Introduction

Hello, I'm Ushii.
Last time, the NIC was recognized as an “InfiniBand controller”, and I ended up stumbled.
I want to build a 100GbE, Ethernet environment, so I need to recognize the NIC as an “Ethernet controller”.
When a person is detected, it is a device that generates an alarm with sound and light.
This time, I would like to introduce the procedure to recognize the NIC as “Ethernet controller”.

Mellanox NIC mode change

Setting up a NIC for use in an Ethernet environment is fairly straightforward. There are only three main steps:

[1] Installation of Mellanox NIC driver software (MLNX-OFED)
[2] Change NIC mode
[3] Restart the server

Let's take a closer look at each!


[1] Installing MLNX-OFED
First, install the driver software (MLNX-OFED) for Mellanox NIC, including the firmware (FW) required for NIC mode conversion. (Driver types and details are described in the wordbook.)
The installation procedure of MLNX-OFED is as follows.


(1) Upload OFED to target server
・Download the OFED tgz file from the URL below.
http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers
・Upload the tgz file to the target server using an SCP server, etc.
*I use "WinSCP" from my laptop to upload.


(2) Unzip the tgz file
・Execute the following command;
[root@localhost ~]# tar xvzf MLNX_OFED_LINUX-4.0-1.0.1.0-rhel7.2-x86_64.tgz
*This time, I put the files in the root directory, but if there are files in other directories, use the “cd” command to change to the desired directory.


(3) Change directory and run install script [root@localhost ~]# cd MLNX_OFED_LINUX-4.0-1.0.1.0-rhel7.2-x86_64/
[root@localhost MLNX_OFED_LINUX-4.0-1.0.1.0-rhel7.2-x86_64]# ./mlnxofedinstall
*When you run the install script, the installer starts as shown below and installation begins (Figure 1).

*Installing MLNX-OFED automatically updates the NIC firmware.
(can be excluded in installation options)

[2] Change NIC mode
Next, use the “Mellanox Firmware Tool (MFT)” included in MLNX-OFED to change the NIC to Ethernet mode (InfiniBand → Ethernet).

(1) Starting MFT
To use MFT, you need to start “Mellanox Software Tool (MST)”. (I won't go into detail here)

Execute the following command to start MST.
[root@localhost ~]# mst start

If you see “Success” as shown in Figure 2, MST started successfully.

(2) Check the ID of the NIC and check the information
ID verification;
[root@localhost ~]# ibv_devinfo | grep vendor_part_id

From Figure 3, we can see that the ID of the NIC is 4115.

(3) Switch to Ethernet mode Just run the following command!
[root@localhost ~]# mlxconfig -d /dev/mst/mt4115_pciconf0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2
P1 and P2 in the above command represent ports, and switching each port to Ethernet mode by setting P1=2 and P2=2.
* Set to "1" if you want to use InfiniBand mode.

[3] Restart the server
Restart the server and you're all set!

After rebooting, the “Ethernet controller If you check if it is recognized as ”,
(Command: [root@localhost ~]# lspci | grep Mellanox )

As you can see from Figure 5, it was properly recognized as “Ethernet controller”! !

Also, I tried connecting two servers directly to see if I could link up with a 100GbE band!
Use the following command to check.
[root@localhost ~]# ethtool ens3f0 (where ens3f0 is the interface name of the NIC)

From Figure 6, we were able to confirm that the link was up at 100GbE!

The server is now ready for building a 100GbE network environment!

Now this issue is resolved!


For details on how to convert the mode of the NIC introduced in this article, please refer to the Mellanox user manual below.
・ MFT_user_manual_4_6_0.pdf

in conclusion

This time, I wrote an article about the settings after installing the NIC on the server.
The user manual at the end of the article is only available in English and has a lot of pages. . .
Combined with that, I struggled with the following points this time. . .

・Finding where the description is
→ It was described in two manuals, and it was difficult just to find it...
・Read and understand the contents of the manual
→Because it is expressed simply in English, there were many requests for a more detailed explanation.

I think there are some people who don't want to read the manual just because it's in English.
(I was also rejected at first lol)
Therefore, I would appreciate it if you could refer to my article when using Mellanox products.

Next time, we plan to post an article about Mellanox switches, so please continue to read the blog. thank you!

Finally, I would like to introduce “Mellanox,” which I am in charge of.


◆ Overview of Mellanox ◇ Headquarters: Yorknam (Israel), Sunnybell (USA)
◇A leading company in broadband, low-latency interconnects
・EDR 100Gbps InfiniBand, 10/25/40/50/56/100 Gigabit Ethernet
・Significantly reduced application data processing time
・Dramatically improve the ROI of the data center service infrastructure