Question

Has anyone noticed their LACP bonds are not balanced across the interfaces for hyperscale.

  • 6 March 2023
  • 1 reply
  • 169 views

Badge +2

We’ve noticed that while using LACP mode 4  on our dell r740xd2 hyerperscales that the interfaces have unbalanced amount of traffic if you do IFCONFIG.  I have p1p2 bonded with p5p2 for the storage network.  P1p1 and p5p1 on the data traffic network.  Notice my rx packets and tx packets are very unbalanced.

p1p1 tx packets are at 410Gib and its partner p5p1 is at 11TiB for example.  

 

Does anyone see the same on their LACP config or solved this issue?  We see the same behavior on a dell 48 port switch and the cisco 9k using Cisco ACI.  Also on hyperscale 1.5 and hyperscale x deployments.

 

p1p1: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether bc:97:e1:2c:9b:00  txqueuelen 1000  (Ethernet)
        RX packets 11213468144  bytes 14318195329557 (13.0 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1279302179  bytes 440269032106 (410.0 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p1p2: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether bc:97:e1:2c:9b:01  txqueuelen 1000  (Ethernet)
        RX packets 7873700537  bytes 9617619985802 (8.7 TiB)
        RX errors 1  dropped 0  overruns 0  frame 59
        TX packets 28099434107  bytes 32698396187309 (29.7 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p5p1: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether bc:97:e1:2c:9b:00  txqueuelen 1000  (Ethernet)
        RX packets 9995455993  bytes 13642001835485 (12.4 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10186853507  bytes 12253200378182 (11.1 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p5p2: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether bc:97:e1:2c:9b:01  txqueuelen 1000  (Ethernet)
        RX packets 24459306896  bytes 29464733404296 (26.7 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11148  bytes 1381894 (1.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 


1 reply

Userlevel 3
Badge +10

I have never seen a properly balanced LACP link ever. In any scenario, Window, Linux, Unix.

In fact I have never heard someone explain exactly how to force them to be balanced. Typically rather than getting double the bandwidth and redundancy, you just Get redundancy. 

 

I am NOT a networking guy but every time I have ever gone down the rabbit hole with any networking team I have ever worked with I have not gotten a straight answer on what is needed to get this working. 

 

does this help you in any way? 

Nope. It’s just a pet peeve.

Reply