Introduction
We conducted an interconnection test between a white Box switch (WBS) and a Cisco switch product, and in this article we will introduce the settings and test results.
By interconnecting with major Cisco products as network switches, you will be able to understand the quality of WBS.
In this interconnection verification, we imagined a data center network and built a small Leaf-Spine configuration using L2 VxLAN/EVPN.
Simple design information
We will summarize simple design information for interconnection verification.
Used equipment
The equipment used is as follows.
| SpineSW | LeafSW-1 | LeafSW-2 | |
|---|---|---|---|
|
Vendor |
Edegecore |
Edgecore |
Cisco |
|
Platform |
AS7726-32X |
AS7326-56X |
C9300-48T |
|
ASIC |
Trident III |
Trident III |
(not clear) |
|
NOS (Ver) |
ecSONiC |
ecSONiC |
IOS |
Configuration information
The configuration information is as follows.
Settings
The settings are summarized below.
| SpineSW | LeafSW-1 | LeafSW-2 | |
|---|---|---|---|
|
VLAN |
No setting (single NW) |
×2 (vlan10、vlan20) |
×3 (vlan10、vlan20、vlan100) |
|
LACP |
Connection with LeafSW-1 and LeafSW-2 |
Connection with SpineSW |
Connection with SpineSW |
|
BGP (iBGP) / EVPN |
|||
|
AS |
65100 |
65100 |
|
|
Router ID |
2.2.2.2 /32 |
3.3.3.3 /32 |
|
|
neighbor |
10.0.0.3 |
10.0.0.2 |
|
|
IPv4 unicast notification |
|||
|
notification address |
2.2.2.2 /32 |
3.3.3.3 /32 |
|
|
EVPN |
|||
|
neighbor |
10.0.0.3 |
10.0.0.2 |
|
|
VNI notification |
All VNIs |
All VNIs |
|
|
VxLAN settings |
|||
|
VTEP |
2.2.2.2 |
3.3.3.3 |
|
|
VLAN-ID / VNI linking |
vlan10 / VNI 10100 |
vlan10 / VNI 10100 |
SpineSW operates as a simple layer 2 switch.
LeafSW-1 and LeafSW-2 set vlan10 and vlan20 in the overlay.
Set up L2VxLAN/EVPN on the underlay and tunnel communication for vlan10 and vlan20.
In addition, in the physical layer, by setting LACP, the communication route with SpineSW is duplicated and the bandwidth is expanded.
Setting command
This is a summary of the commands actually set on the switch.
*It is assumed that the management functions and interface speed settings have been completed and will be omitted.
(Host name, management account, management IP, SSH/Telnet, speed settings, Breakout settings, etc.)
You can download the configuration commands below.
SpineSW settings
As a premise, the interface names and physical ports of AS7726-32X correspond as shown below.
| interface name | physical port number |
|---|---|
|
Ethernet112 |
Port29 |
|
Ethernet116 |
Port30 |
|
Ethernet124 |
Port32 (Breakout lane 1) |
|
Ethernet125 |
Port32 (Breakout lane 2) |
Configure the following settings for SpineSW.
Configuring LACP
LeafSW-1 settings
As a premise, the interface names and physical ports of AS7326-56X correspond as shown below.
| interface name | physical port number |
|---|---|
|
Ethernet0 |
Port1 |
|
Ethernet11 |
Port2 |
|
Ethernet124 |
Port49 |
|
Ethernet125 |
Port50 |
Configure the following settings for LeafSW-1.
The details of each setting are summarized below.
VLAN settings
Configuring LACP
IP address settings
(Used as VTEP, BGP router ID)
Configuring VxLAN
### VTEP configuration
admin@LeafSW-1:~$ sudo config vxlan add vtep 2.2.2.2
### NVO creation, linking with vtep (2.2.2.2)
admin@LeafSW-1:~$ sudo config vxlan evpn_nvo add nvo vtep
### VLAN-VNI mapping in NVO
admin@LeafSW-1:~$ sudo config vxlan map add vtep 10 10100
admin@LeafSW-1:~$ sudo config vxlan map add vtep 20 10200
BGP/EVPN settings
*In ecSONiC, configure BGP settings using FRRouting.
### FRRouting Configuration mode > Transition to Config mode
admin@LeafSW-1:~$ vtysh
LeafSW-1# configure terminal
### Create BGP (AS 65100) and go to configuration mode
LeafSW-1(config)# router bgp 65100
### Set Router ID to 2.2.2.2
LeafSW-1(config-router)# bgp router-id 2.2.2.2
### Set 10.0.0.3 / AS 65100 [LeafSW-2 information] as BGP neighbor
LeafSW-1(config-router)# neighbor 10.0.0.3 remote-as 65100
### Set address information to notify BGP neighbors
LeafSW-1(config-router)# address-family ipv4 unicast
### Advertise 2.2.2.2 (your VTEP)
LeafSW-1(config-router-af)# network 2.2.2.2/32
### Transition from address family mode
LeafSW-1(config-router-af)# exit-address-family
### Go to EVPN configuration mode
LeafSW-1(config-router)# address-family l2vpn evpn
LeafSW-1(config-router-af)# neighbor 10.0.0.3 activate
### Advertise all VNIs
LeafSW-1(config-router-af)# advertise-all-vni
### Transition from address family mode
LeafSW-1(config-router-af)# exit-address-family
LeafSW-1(config-router)# exit
LeafSW-2 settings
Configure the following settings for LeafSW-2.
The details of each setting are summarized below.
Interface settings
*vlan100 is the VLAN for connecting to SpineSW and LeafSW-1
IP address setting
(Used as router ID for VTEP and EVPN/BGP)
Configuring EVPN
▶Set to Ingress replication
▶Specify router ID to Loopback0
▶VNI notification settings
▶VxLAN usage settings, Ingress replication settings
Associate EVPN instance with VLAN and configure VNI
Configuring the NVE interface
▶Specifying settings for BGP use
▶Linking Loopback0, VNI 10100, 10200, Ingress-Replication usage settings
Configuring BGP
![[11th] Thumbnail image of how to create a network visualization dashboard using Grafana for beginners](/business/network/columns/071625e009bd21fc48547707c0326d8a_1.png)
![[10th] Thumbnail image of SNMP data visualization method using OSS ~Grafana x OcNOS~](/business/network/columns/2d131c51f152619d14c0af446a1ddc71_1.png)
![[Part 8] Implementation example using P4 language and FPGA Smart NIC ~NAT version~ Thumbnail image](/business/network/columns/9799fbf29e361c08287995348dafacbe.png)
Communication test/status confirmation
Test results
As a result of flowing traffic between each terminal, we were able to confirm that the terminals in the same VLAN (terminals A to C and terminals B to D) were able to communicate with each other as shown below.
| address | |||||
|---|---|---|---|---|---|
|
sender |
Terminal A (vlan10) | Terminal B (vlan20) | Terminal C (vlan10) | Terminal D (vlan20) | |
|
Terminal A (vlan10) |
- (separate VLAN) |
〇 |
- (separate VLAN) |
||
|
Terminal B (vlan20) |
- (separate VLAN) |
- (separate VLAN) |
〇 |
||
|
Terminal C (vlan10) |
〇 |
- (separate VLAN) |
- (separate VLAN) |
||
|
Terminal D (vlan20) |
- (separate VLAN) |
〇 |
- (separate VLAN) |
||
Check status
Check the status of each configured item.
Interface information
You can check the interface information using the following command.
ecSONiC/IOS common: show interfaces status
LACP information
LACP information can be checked using the following command.
ecSONiC： show interfaces portchannel
IOS： show etherchannel summary
VLAN information
VLAN information can be checked using the following command.
ecSONiC: show vlan brief or show vlan config
IOS: show vlan
BGP
BGP information can be checked using the following command.
ecSONiC / IOS common: show ip bgp
Route information
You can check the route information using the following command.
ecSONiC / IOS common: show ip route
VLAN-VNI mapping
You can check the VLAN-VNI mapping status using the following command.
ecSONiC： show vxlan vlanvnimap
IOS： show nve vni
Peer VNI information
You can check the peer's VNI information using the following command.
ecSONiC： show vxlan remotevni <peer-IP>
IOS： show l2vpn evpn peers vxlan
Other notes
Unable to send BUM traffic over VxLAN with Ingress Replication from Catalyst switch
There are two types of VxLAN BUM transfer methods: Underlay IP Multicast and Ingress Replication (EVPN).
For the following reasons, the VxLAN BUM transfer method must be ingress replication, and BUM cannot be transferred from a Catalyst switch.
● BUM cannot be transferred from Catalyst switches using Ingress Replication.
● ecSONiC (as of ec202111.4) does not support Underlay IP Multicast.
*BUM: B roadcast, Unknown Unicast, Multicast
*With Cisco's Nexus series (data center switches), BUM transfer is also possible with Ingress Replication.
In the case of Unknown Unicast, it can be avoided by sending the packet from under the Remote VTEP (=LeafSW-1 in the test configuration) as seen from the C9300.
This is because by starting communication from the opposite side, it will be added to the C9300's VxLAN MAC address table.
We assume that there is currently no workaround for Broadcast and Multicast.
Summary
This time, we were able to build an L2 VxLAN/EVPN configuration between WBS (ecSONiC) and Cisco switch products.
We now know the settings required for WBS and Cisco switches for interconnection.
Additionally, by being able to interconnect the WBS and Cisco switches using L2 VxLAN/EVPN, I was able to understand once again the high quality of the WBS.
Next, we will try interconnection using L3 VxLAN/EVPN.
Settings file download
The configuration file used in this article can be downloaded from the URL provided in the information email by clicking "Apply here" below.
Please try.
At the end
Macnica offers a service that provides a remote environment where you can actually experience open networking and conduct testing and verification.
This service allows us to verify the operability of the network OS and perform tests that combine network OSs, white Box switches, and optical transceivers from various manufacturers.
Remote verification service image diagram
It is a service that allows you to easily test open networking, and the basic configuration is free of charge.
You can check the downloadable materials for available network OS and white Box switches, specific use cases, and how to apply. The materials can be downloaded from the URL listed in the information email by answering the questionnaire from the "Macnica Network OS Remote Verification Service" below.
These people are using the remote verification service.
Here are some testimonials from people who have actually used the service.
Furukawa Network Solution Co., Ltd.
“Recently, the number of remote service environments is increasing, but I was impressed with the ease of access to the evaluation equipment.
The materials you provided were easy to understand, and we were able to proceed smoothly with the intended verification. ”
Related information
![[Part 5] What is Edgecore SONiC? Thumbnail image of ~Explaining the overview and features of the commercial version of SONiC~](/business/network/columns/34b7d7142d053b12b46b146e025d2fcd.png)
![[Part 9] SNMP visualization of white Box switches using SONiC - Grafana x InfluxDB - Thumbnail image](/business/network/columns/84d7a7525020782725001d27311eddde_1.png)
![[Part 13] Thumbnail image of network visualization of white Box switches using commercial tools ~Aviz ONES w/SONiC edition~](/business/network/columns/8775dea6143af666ecc189f0b3a5f442_2.png)
Click here for list of materials
In addition to introducing products handled by Macnica,
We publish materials related to open networking, such as BGP cross network automatic construction files and network operation test evaluation reports.
Click here for details
Product Page Top
Edgecore Networks
We continue to be a pioneer in open networking by developing and selling products related to OpenNetworking/white Box switches.
DELTA ELECTRONICS
We have many achievements in various fields such as networks, IoT, and electronic components.
CGS Tower Networks
We provide a network packet broker (NPB) that utilizes state-of-the-art general-purpose hardware.
Inquiry
If you have any questions regarding this article, please contact us below.
Macnica
In charge of OpenNetwork
Contact us by phone:
045-470-9831
Email us:
projectmonstar@macnica.co.jp