Linux dual network card binding load balancing details

Linux dual network card binding to achieve load balancing, for your reference, the specific content is as follows

System environment: CentOS release 6.9 (Final)Linux centos6 2.6.32-696.10.1. es9EN6.x86_64

Ubuntu system USES ifenslave for network card configuration, the method is similar.

Close the NetworkManager

# service NetworkManager stop
# chkconfig NetworkManager off

Create the bond0 interface configuration file

# cd /etc/sysconfig/network-script/
# vim ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.100.106
PREFIX=24
GATEWAY=192.168.100.1
DNS1=202.103.24.68
BOOTPROTO=static
ONBOOT=yes
USERCTL=no

Modify the configuration of eth0 and eth1

# vim ifcfg-eth0 
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

# vim ifcfg-eth1 
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

Load the bonding module

# vim /etc/modprobe.d/bonding.conf　　# new bonding.conf file 
alias bond0 bonding
options bond0 miimon=100 mode=6

Description:

miimon is used for link monitoring. For example: miimon=100 in milliseconds. This 100 right here is 100ms, which is 0.1 seconds. Every 100ms, the system monitors the state of one link connection. If one line is not working, the system will transfer to another line. mode refers to the working mode, and there are 7 kinds (0-6). These two parameters can be configured in the/etc/modprobe d/bonding conf file, namely options bond0 miimon = 100 mode = 6, or write directly in the configuration file of bond0, namely BONDING_OPTS = "mode = 6 miimon = 100"

Restart network service

# service network restart

View the status of the bond0 interface

[root@centos6 ~]# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 52:54:00:a9:4e:2c
Slave queue ID: 0

Slave Interface: eth1
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 52:54:00:91:1b:68
Slave queue ID: 0

If the bond configuration has been modified multiple times, you may need to restart the system before the bond0 interface status is updated.

Check the status of network interface transceiver

# watch -n 1 ifconfig -a

Introduction to 7 working modes of bond

1. mode=0(ES77en-ES78en)(balanced swing cycle strategy)

Link load balancing, increased bandwidth, support fault tolerance, 1 link failure will automatically switch the normal link. Switches need to be configured with aggregators. Cisco calls them port channel.
Features: packet transmission sequence is sequential transmission (i.e., the first packet to eth0, the next packet to eth1... 1. The cycle continues until the last transmission is completed). This mode provides load balancing and fault tolerance. However, if a packet of a connection or session is sent from a different interface and goes through a different link halfway, the unordered arrival of the packet at the client side is likely to occur, and the unordered arrival of the packet needs to be sent again, so that the throughput of the network will decrease

2. mode=1(active-ES90en)(master-backup strategy)

This is the primary and standby mode. Only one network card is active and the other is the standby standby. All traffic is processed on the active link.
Features: Only 1 device is active, and when one is down and the other is immediately converted from backup to primary. The mac address is externally visible, and the MAC address of bond is only 1 from the outside to avoid confusion with switch(switch). This pattern only provides fault tolerance; It can be seen that the advantage of this algorithm is that it can provide high availability of network connections, but its resource utilization ratio is low, with only one interface in working state. In the case of N network interfaces, the resource utilization ratio is 1/N

3. mode=2(ES106en-xor)(Balanced strategy)

Represents XOR Hash load sharing, in conjunction with the converged mandatory non-negotiation mode of the switch. (xmit_hash_policy is required, and switch configuration port channel is required)
Features: Transmits packets based on the specified transport HASH policy. The default policy is :(source MAC address XOR destination MAC address) % slave number. Additional transport policies can be specified through the xmit_hash_policy option, which provides load balancing and fault tolerance

4. mode=3(broadcast)(Broadcast strategy)

Means that all packets are emitted from all network interfaces, an unbalanced, redundant mechanism, but too wasteful of resources. This model is applicable to the financial industry, because they need a highly reliable network and do not allow any problems. Need to cooperate with the converged mandatory non - negotiation mode of the switch.
Features: Each packet is transmitted on each slave interface. This mode provides fault tolerance

5, mode=4(802.3ad)(IEEE 802.3ad Dynamic link aggregation)

Support for the 802.3ad protocol, in conjunction with the converged LACP mode of the switch (xmit_hash_policy). The standard requires that all devices perform converged operations in the same rate and duplex mode, and that no connection USES more than one interface bandwidth, as with bonding load balancers other than balance-ES148en mode.

Features: Create 1 aggregation group that shares the same rate and duplex Settings. According to the 802.3ad specification, multiple slave work under the same active aggregate.
The slave election for outbound traffic is based on the transport hash policy, which can be changed from the default XOR policy via the xmit_hash_policy option. It should be noted that not all transport policies are adapted to 802.3ad, especially considering the packet disorder mentioned in section 43.2.4 of the 802.3ad standard. Different implementations may have different adaptations.

Requirements:

Condition 1: ethtool supports getting the rate and duplex Settings for each slave
Condition 2: switch(switch) supports IEEE 802.3ad Dynamic link aggregation
Condition 3: Most switch(switches) require a specific configuration to support the 802.3ad mode

6. mode=5(balance-tlb)(Adapter transmission load balancing)

slave is selected for sending based on the load of each slave, and the current slave is used for receiving. This mode requires some ethtool support for network device drivers with slave interface. And the ARP monitoring is not available.

Features: No special switch(switch) support for channel bonding is required. Allocate outgoing traffic on each slave based on the current load (calculated according to speed). If the slave receiving data fails, another slave takes over the MAC address of the failed slave.
Requirements: ethtool supports getting the rate of each slave

7. mode=6(balance-alb)(adapter adaptive load balancing)

rlb(receive load balancer receive load balance) was added to tlb of 5. No switch(switch) support is required. Receiving load balancing is achieved through ARP negotiation.

Features: This mode includes ES225en-ES226en mode, plus receive load balancing for IPV4 traffic (receive load balance, rlb) and does not require any switch(switch) support. Receiving load balancing is achieved through ARP negotiation. The bonding driver intercepts the ARP reply sent by the native and rewrites the source hardware address to the only 1 hardware address of one of the bond, enabling different counterparts to communicate using different hardware addresses. Incoming traffic from the server side is also balanced. When the native sends the ARP request, the bonding driver copies and saves the IP information on the opposite side from the ARP package. When the ARP reply arrives from the opposite end, the bonding driver extracts its hardware address and issues an ARP reply to one of the bond.

One problem with load balancing using ARP negotiation is that the hardware address of bond is used every time an ARP request is broadcast, so when the other end learns the hardware address, all the received traffic will flow to the current slave. This problem can be solved by sending updates (ARP replies) to all the counterparties, which contain their unique hardware addresses, causing traffic to be redistributed.

Receive traffic is also redistributed when a new slave is added to bond or when an inactive slave is reactivated. The received load is distributed sequentially (round robin) on slave, the highest speed in bond
When a link is reconnected or a new slave is added to bond, the receiving traffic is reallocated across all currently active slave, initiating an ARP response to each client using the specified MAC address. The updelay parameter described below must be set to a value greater than or equal to the switch(switch) forwarding delay to ensure that the ARP reply sent to the opposite end is not blocked by switch(switch).

Requirements:

Condition 1: ethtool supports getting the rate of each slave;
Condition 2: The underlying driver supports setting the hardware address of a device so that there is always an slave(curr_active_slave) using bond's hardware address, while ensuring that each slave in bond has a unique hardware address. If curr_active_slave fails, its hardware address will be taken over by the newly selected curr_active_slave which is the difference between mod=6 and mod=0: mod=6, first take up eth0 traffic, then eth1... . ethX; If mod=0, we will find that the flow rate of both ports is very stable, and the basic bandwidth is 1 kind. While mod=6, you will find that the first port has a high flow rate and the second port only accounts for a small part of the flow rate.

mode5 and mode6 do not require switch side Settings, network CARDS can be automatically aggregated. mode4 requires support for 802.3ad. mode0, mode2 and mode3 theoretically require static aggregation.

However, in the actual measurement, mode0 can receive unevenly without setting the switch by means of mac address spoofing.