Question

Help with Strange Gatway Behavior

  • 7 September 2023
  • 7 replies
  • 56 views

Badge +1

Here is a problem we are having with media agents at one location.  We are seeing backups and restores with wildly different performance for any gateway installed on a Nutanix cluster.  An adjacent older VMware cluster has no problems.  The two clusters share the same 1G Internet pipe at a colo.  

We have another Nutanix cluster at a different location that has pretty decent performance consistently.  We cannot find any one thing that might explain the behavior, and the poor and inconsistent performance makes managing the platform very difficult at that site.  

We would like to understand why the gateway running on the older cluster (HPE servers and EMC storage) which runs on 1G interconnects, is orders of magnitude faster than some of the gateways running the new Nutanix cluster (HPE DX-380’s) with 10G interconnects.  

Running ordinary uploads and downloads on any of the Nutanix-hosted gateways is fast, as expected.  So, it doesn’t seem to be Internet, or hardware related.  It smells as though Metallic software is misbehaving, but only on some gateways on that cluster.

We have been escalating with the Metallic support but so far there have been no revelations!  Our next step may be to install gateways on physical hardware and see what happens, taking the virtual environment out of the picture as far as possible. 

Has anyone else worked a problem like this with their environment?   


7 replies

Userlevel 3
Badge +8

Good day @bmduncan33 ,

Thanks for reaching out on the Metallic community. It’s hard to say directly depending on environmental variables however if you can provide me your Support case number, I’ll review and work to get the right eyes and priority on it.

 

Thanks

Michael

Badge +1

230810-704

Userlevel 3
Badge +5

Do you have AV exclusions in place Antivirus Exclusions for Windows (commvault.com)?

 

Are there any firewalls in place, packet inspecting software or something like this?

Badge +1

Thanks for the response.  The only thing that is consistent with this behavior is that all the affected gateways are running on one Nutanix cluster.  I don’t know about the AV exclusions but will read that link.  Yes to firewalls, but no to packet inspection like wireshark, etc.  The same firewalls are being used by the gateways not running on that cluster and there is no impact to them. 

We are hoping that Metallic engineers can help us find something in logs, but so far there’s been very little traction.  We may have to deploy two physical servers to run the gateway software outside that cluster, but that seems like an extreme choice.

Userlevel 3
Badge +5

Maybe you have network topology in place for old environment, but not for new one? AV exclusions might be helpful too. 

 

 

Badge +1

I’m not clear what you mean about network topology.  Can you elaborate on that please?

Userlevel 3
Badge +5

I’m not clear what you mean about network topology.  Can you elaborate on that please?

Sorry, we are talking about Metallic here and it’s Commvault feature to force traffic into tunnels to i.e. to allow traffic only via open network ports. 

 

 

Reply