MLNX_OFED provides multiple utilities for debugging device status and connection status within the InfiniBand fabric. This time we will introduce some of them.

ibdiagnet

Commentary

It scans the fabric and extracts all available information about connectivity and devices.
Generate the following files in the output directory. (The directory is determined by the -o option.)

Usage

ibdiagnet [-c ] [-v] [-r] [-o ] [-t ] [-s ] [-i ] [-p ] [-wt] [-pm] [-pc] [-P <= >] [-lw <1x|4x|12x>] [-ls <2.5|5|10>] [-skip ] [-load_db ]
output file Commentary
ibdiagnet.log a dump of all reports generated by the given flag
ibdiagnet.lst List of all nodes, ports and links in the fabric
ibdiagnet.fdbs Dumping the unicast forwarding table of a fabric switch
ibdiagnet.mcfdbs Dump multicast forwarding table on fabric switch
ibdiagnet.masks Include duplicate records if port/node GUIDs are duplicated
ibdiagnet.sm List all subnet managers (state and priority) in the fabric
ibdiagnet.pm Dump fabric link performance counter values
ibdiagnet.pkey A dump of the existing partitions and the hosts they belong to
ibdiagnet.mcg Dumping multicast groups, their properties and host port members
ibdiagnet.db Dumping the internal subnet database

ibv_devices

Commentary

Show available InfiniBand devices from user space (including Node GUID).

Usage

ibv_devices

Example) Display the names of all available InfiniBand devices.

ibv_devinfo

Commentary

Query InfiniBand devices and display information about devices available from user space.

Usage

ibv_devinfo [-d] [-i] [-l] [-v]
Example) Display the names of all available InfiniBand devices.
Example) Query the device mlx4_0 and display the information of Port2.

ib status

Commentary

Shows basic information from the InfiniBand driver of the local node. Outputs include LID, SMLID, port state, port physical state, port signal width, port speed.

Usage

ibstatus [-h] [[:]]*
ex) Display all available devices and their port states.

ibportstate

Commentary

Query physical port and logical link states for InfiniBand ports. It also adjusts the speed of the enabled InfiniBand ports.

Usage

ibportstate [-d] [-e] [-v] [-V] [-D] [-G] [-s] \

[-C] [-P] [-t] ¥

[] [ []]

ex) Display all available devices and their port states.

ib route

Commentary

Display the forwarding table using SMP for a specific switch LID and optional lid (mlid) region.

Usage

ibroute [-h] [-d] [-v] [-V] [-a] [-n] [-D] [-G] [-M] [-s] \

[-C] [-P] [-t] ¥

[ [ []]]

Example) Dump the lids of all valid ports connected to the lid 2 switch.

perfquery

Commentary

Queries performance and error counters for InfiniBand ports. You can also reset the counter after reading it, or simply reset it.

Usage

perfquery [-h] [-d] [-G] [-a] [-l] [-r] [-C] [-P] [-R]

[-t] [-V] [[[port][reset_mask]]]

Example) Read the performance counter of the port of the own node

ibcheckerrs

Commentary

Validate InfiniBand ports and report errors in counters that exceed thresholds.

Usage

ibcheckerrs [-h] [-b] [-v] [-G] [-T ] [-s] [-N |-nocolor] [-C ca_name] [-P ca_port] [-t timeout_ms] []
Example) Verify the node counter of LID 2.

ibhosts / ibswitches

Commentary

(ibhosts) Extract CA nodes in the InfiniBand subnet topology.
(ibswitches) Extract switches in the InfiniBand subnet topology.

Usage

ibhosts [-h] [] | -C ca_name –t(timeout)

ibswtiches [-h] [] | -C ca_name –t(imeout)

ibdump (supported since MLNX_OFED 1.5.1)

available hardware

Mellanox ConnectX / ConnectX-2 Adapter Device

Commentary

Dumps InfiniBand traffic sent and received on Mellanox ConnectX/ConnectX-2 adapter InfiniBand ports. The dump file can be loaded by Wireshark and visually analyzed for traffic.

Usage

ibdump [options]

Operation method for HCA sniffing
- Run ibdump with options
・Run the application for the traffic you want to analyze
Stop ibdump ( CTRL-c ) or wait until the data buffer is full
Open Wireshark and load the generated file

*Wireshark can be downloaded from www.wireshark.org for the current release. See the ibdump_release_note.txt file for details.

ex) run ibdump
ex) run ibdump

The above information is extracted from Mellanox_OFED_Linux_user_manual, if you want to check the details
You can get it from http://www.mellanox.com/ > Downloads > InfiniBand/VPI SW/Drivers > MLNX_OFED. Or refer to the manual from Linux using the man command.