What is teaming?
Teaming or LACP (802.3ad) is a technique used to bond together multiple interfaces to achieve higher combined bandwith. NOTE: every clients speed can only be as high as the single link speed of one of the members. That means, if the interfaces I use in the bond have 1 Gigabit, every client will only have a maximum speed of 1 Gigabit. The advantage of teaming is, that it can handle multiple connections with 1 Gigabit. How many connections depends on the amount of your network cards.
I'm using 2 network cards for this team on my server. That means I can handle 2 Gigabit connections at full rate on my server provided the rest of the hardware can deliver that speed.
There also exists 'Bonding' in the Linux world. They both do the same in theory but for a detailed comparison check out this article about teaming in RHEL7.
To create a teaming-interface, we will first have to remove all the interface configurations we've done on the (soon to be) slave-interfaces for our team. I've edited the ifcfg-files to look like the following:
Teaming or LACP (802.3ad) is a technique used to bond together multiple interfaces to achieve higher combined bandwith. NOTE: every clients speed can only be as high as the single link speed of one of the members. That means, if the interfaces I use in the bond have 1 Gigabit, every client will only have a maximum speed of 1 Gigabit. The advantage of teaming is, that it can handle multiple connections with 1 Gigabit. How many connections depends on the amount of your network cards.
I'm using 2 network cards for this team on my server. That means I can handle 2 Gigabit connections at full rate on my server provided the rest of the hardware can deliver that speed.
There also exists 'Bonding' in the Linux world. They both do the same in theory but for a detailed comparison check out this article about teaming in RHEL7.
To create a teaming-interface, we will first have to remove all the interface configurations we've done on the (soon to be) slave-interfaces for our team. I've edited the ifcfg-files to look like the following:
The first interface:
[archy@server ~]$ sudo cat /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
NAME=eth0
UUID=811289b3-8c1f-4f38-8885-21fa8aef107f
DEVICE=eth0
ONBOOT=no
The second interface: [archy@server ~]$ sudo cat /etc/sysconfig/network-scripts/ifcfg-eth1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
NAME=eth1
UUID=811289b3-8c1f-4f38-8885-21fa8aef107f
DEVICE=eth1
ONBOOT=no
The teaming-interface: [archy@server ~]$ sudo cat /etc/sysconfig/network-scripts/ifcfg-team0
DEVICE=team0
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=172.31.0.250
PREFIX=24
GATEWAY=172.31.0.254
DNS1=127.0.0.1
DOMAIN="archyslife.lan"
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
NAME=team0
UUID=993942fd-9aee-4544-9ad7-8bedcd7beb0b
ONBOOT=yes
DEVICETYPE=Team
TEAM_CONFIG="{\"runner\": {\"name\": \"lacp\", \"active\": true, \"fast_rate\": true, \"tx_hash\": [\"eth\", \"ipv4\", \"ipv6\"]},\"link_watch\": {\"name\": \"ethtool\"}}"
Now it's time to restart the network.service to activate the changes
[archy@server ~]$ sudo systemctl restart network.service
The state of the teaming can be checked using the teamd-tools:
[archy@server ~]$ sudo teamdctl team0 state
setup:
runner: lacp
ports:
eth0
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
down count: 0
runner:
aggregator ID: 3, Selected
selected: yes
state: current
eth1
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
down count: 0
runner:
aggregator ID: 3, Selected
selected: yes
state: current
runner:
active: yes
fast rate: yes
[archy@server ~]$ sudo teamnl team0 ports
3: eth0: up 1000Mbit FD
2: eth1: up 1000Mbit FD
The easiest way to configure is to use the nmcli or nmtui tool provided by the NetworkManager-package.
[archy@server ~]$ sudo nmcli connection add type team con-name team0 ifname team0 config '{"runner": {"name": "lacp"}}'
[archy@server ~]$ sudo nmcli connection add type team-slave con-name team0-eth0 ifname eth0 master team0
[archy@server ~]$ sudo nmcli connection add type team-slave con-name team0-eth1 ifname eth1 master team0
Usually the nmcli should have brought up the connection(s) automatically, but I usually restart the network.service anyway,. Just to make sure.
[archy@server ~]$ sudo systemctl restart network.service
The state of the teaming can be checked using the teamd-tools:
[archy@server ~]$ sudo teamdctl team0 state
setup:
runner: lacp
ports:
eth0
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
down count: 0
runner:
aggregator ID: 3, Selected
selected: yes
state: current
eth1
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
down count: 0
runner:
aggregator ID: 3, Selected
selected: yes
state: current
runner:
active: yes
fast rate: yes
[archy@server ~]$ sudo teamnl team0 ports
3: eth0: up 1000Mbit FD
2: eth1: up 1000Mbit FD
Feel free to comment and / or suggest a topic.
i follow process but i have :
ReplyDelete8: team0: mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether xx.xx.xx.xxx.x2 brd ff:ff:ff:ff:ff:ff
This might be caused by different reasons.
DeletePlease note that you will need to configure the switch / server on the other site of the link with 802.3ad (LACP), otherwise you've created a networking loop.
Also this procedure might change if you are doing this on VMs.
I have tried to configure this LACP team at least 10 different ways And I cannot get the team to come up and the CISCO ports stay in suspend. and I missing something?
ReplyDeletePROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=10.250.91.35
PREFIX=24
GATEWAY=10.250.91.1
DNS1=10.250.40.102
DOMAIN=dev.paytel.com
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=Team0
UUID=aebbbeaa-5116-4f2e-87dc-e9774bfac23e
DEVICE=nm-team
ONBOOT=yes
DEVICETYPE=Team
TEAM_CONFIG="{\"runner\": {\"name\": \"lacp\", \"active\": true, \"fast_rate\": true, \"tx_hash\": [\"eth\", \"ipv4\", \"ipv6\"]},\"link_watch\": {\"name\": \"ethtool\"}}"
NAME=Slave1
UUID=20ce56af-61c4-47d6-b368-7fe73a097205
DEVICE=em1
ONBOOT=yes
TEAM_MASTER=nm-team
DEVICETYPE=TeamPort
TEAM_MASTER_UUID=aebbbeaa-5116-4f2e-87dc-e9774bfac23e
NAME=Slave2
UUID=e9760bca-6235-45b7-906b-344e6f8567d4
DEVICE=em2
ONBOOT=yes
TEAM_MASTER=nm-team
DEVICETYPE=TeamPort
TEAM_MASTER_UUID=aebbbeaa-5116-4f2e-87dc-e9774bfac23e
YPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=em1
UUID=3c611987-2ea7-46bc-a5d5-3157bc0d18b1
DEVICE=em1
ONBOOT=no
~
~
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=em2
UUID=b5c363f3-5d83-49c0-a1bc-76e9fdd88fc1
DEVICE=em2
ONBOOT=no
~
Hi there,
Deleteyour config seems to be okay from the first glance. Check your logs and verify the config on the Switche's end.
You could also start over using the nmcli commands and go from there.
Hello Adrian
DeleteYes sir, I have tried that and still no connectivity. My port just won't come out of "suspend", This is my switch configs for my port-channel and the two ports
interface GigabitEthernet 2/0/39
description **OmniCache Test eth0**
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport trunk allowed vlan 2-4024
switchport mode trunk
switchport nonegotiate
channel-group 25 mode active
spanning-tree portfast trunk
end
!
interface GigabitEthernet 3/0/39
description **OmniCache TestTeam eth1**
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport trunk allowed vlan 2-4024
switchport mode trunk
switchport nonegotiate
channel-group 25 mode active
spanning-tree portfast trunk
end
interface Port-channel25
description **OmniCache Team 2/0/39 3/0/39**
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport trunk allowed vlan 2-4094
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
end
Alright, I'm not sure on the 'nonegotiate' but I'm fairly unfamiliar with cisco.
DeleteCould you test your configuration using bonding as well with LACP? Hopefully we find out that way which side is misbehaving.
A howto can be found here: https://archyslife.blogspot.com/2018/05/centos-75-bonding-and-teaming.html
not sure the degree of importance, however i do see you renamed the interfaces to slave... but in the TEAM_CONFIG I see [\"eth\ eth0 probably don't exists
DeleteI know this is a little old but in case anyone else is looking for this
ReplyDeleteMake sure auto negotiate is set to yes for your team interfaces on your server
nmcli con mod team0-eno1 802-3-ethernet.auto-negotiate yes
My interfaces were set to no even though I didn't explicitly define them on initial setup, and my Cisco switch complained that LACP wasn't enabled on the server NICs and would suspend the channel group. This is on CentOS 8.4.