How to change parameters with ethtool

ethtool is used by entering the command "ethtool" on the Linux UART terminal.
Parameters related to the operation of the Ethernet driver can be "displayed" by specifying the -c (lowercase) option on the command line, and "changed" by specifying the -C (uppercase) option. (The parameters handled by the -c / -C options seem to be called coalesce parameters.)

The command line used in this experiment is the following two types.

• Show coalesce parameter (with -c (lowercase))

"ethtool -c <network device name>" (e.g. ethtool -c eth0)

• Change the coalesce parameter (specify -C (uppercase))

"ethtool -C <network device name> <parameter name> <setting value>" (e.g. ethtool -C eth0 rx-usecs 40)

For other operations, refer to the help screen displayed by "ethtool --help".

Among the coalesce parameters, the following parameters seem to affect network performance.

• rx-usecs ... Delay time from packet arrival to reception interrupt issuance (usec)
• tx-usecs ... Delay time from packet transmission to issuance of transmission interrupt (usec)
• tx-frames ... Maximum number of transmit packets until a transmit interrupt is issued

The experiments in this article also edit the above parameters. In addition to the initial setting values (before change), the experiment was conducted in an environment with two patterns of parameter changes (setting change 1 and setting change 2). At that time, the parameter changes for setting change 1 and setting change 2 are the same for both evaluation boards. In addition, it seems that there is a lower limit for the setting value of rx-usecs, but it was possible to set a smaller value by changing the source code of the driver. (implemented in setting change 2)

Refer to the figure below for specific parameter setting values and driver changes.

Experimental result

ping response

I compared the response time (time=XXX ms) displayed when executing the ping command, and it seems that the setting value of rx-usecs greatly affects the ping response. We were able to confirm a significant improvement in response time by reducing rx-usecs.
Separately, I tried changing the value of the delay setting (tx-usecs, tx-frames) on the sending side, but the result was that the response time did not change significantly even if the parameters on the sending side were changed.

iperf throughput

We also compared the throughput measurement results by iperf. Like ethtool, iperf is software that needs to be installed separately, but it is also installed in the Linux software reference environment for SoC FPGA from the beginning, so you can use it as it is.

The iperf results didn't show a clear trend like the ping response time, but it did show that the maximum throughput is affected not only by rx-usecs, but also by the delay setting on the sender side.
Setting change 1 (only rx-usecs was changed to a smaller value than the initial setting) showed an improvement in throughput when the packet size was specified, but setting change 2 (while minimizing rx-usecs, sending side delay If you change the setting to a smaller value), the result was that the throughput decreased. This is probably the result of changing the sender's delay setting to a smaller value.

Summary

The experimental results confirmed that changing the setting of the coalesce parameter in ethtool significantly alters ping response time and iperf results. This suggests that using ethtool (specifically changing the coalesce parameter) can be an effective method when Ethernet performance tuning is required.
If the latency and throughput of Ethernet packet transmission and reception do not meet expectations, first try tuning the coalesce parameters using ethtool.

Finally, when changing the settings of the coalesce parameter using ethtool, please note the following:

Points to note
Settings such as rx-usecs and tx-usecs introduce a certain amount of delay into the Ethernet driver's send and receive processing, preventing packet transmission and reception from negatively impacting the overall Linux system performance. When modifying coalesce parameters to minimize delay, the frequency of send and receive interrupts will increase, so you need to consider the impact on other processes (the entire Linux system) and determine the optimal value.