Redundant configuration with ib-bonding

Bonding can be used with InfiniBand as well as with Ethernet. This ib-bonding works on ipoib and can currently only be used in active / backup mode ( mode=1) (confirmed in MLNX_OFED-1.5.2-2.1.0). The settings are almost the same as for Ethernet.

Write to network script (using RHEL5.5)

1. Add the following to the master (bond0) interface.

TYPE=Bonding MTU=<Match with slave MTU>
  
Example: /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.1.1
NETMASK=255.255.255.0
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
TYPE=Bonding
MTU=65520
 
* MTU=65520 must be set to the same value as all IPoIB slaves must be operating in Connected Mode (CM). When used in Datagram Mode, MTU=2044.

2. Add the following to the slave (ib) interface.

SLAVE=yes
MASTER=
TYPE=InfiniBand
PRIMARY=
 

Example: /etc/sysconfig/network-scripts/ifcfg-ib0

DEVICE=ib0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
TYPE=InfiniBand
PRIMARY=yes

3. Add the following description to /etc/modprobe.conf.

alias bond0 bonding
options bond0 miimon=100 mode=1 max_bonds=1

* max_bonds is the number of bonds. You can check the operation as follows.

ib0 interface failed
Automatically send data on backup ib1

See ib-bonding.txt in the MLNX_OFED-1.5.2-2.10 installation directory for details.