commvault.com commvault.com
Question

CommVault Upgrade 11.28 to 11.32.45 App Crash on Media Agents

  • 23 March 2024
  • 3 replies
  • 99 views
A
Rank
Userlevel 1
Badge +5

Hi Community,
our customer has upgraded the CommVault Environment from Version 11.28.102 to 11.32.45.
Since the upgrade we have Issues with AUXCopies and the VMware VSA backups.

We have this Issues with our 4 physical Media agents (Windows 2019) at the head office.
The following errors occur sporadically on different DDBs and MediaAgents

 

The following app crash events occur in the Windows log on all 4 MediaAgents

 

I have already carried out a CommVault Instance Repair on all 4 media agents, but unfortunately this has not solved the problem.
A CommVault Case was created. Support suspects a Windows OS issue. But I can't believe that all 4 MediaAgents have an OS problem.

A "check readiness" on the MAs shows no problems

Has anyone had similar problems after upgrading CommVault to version 11.32.45?

best regards,
Andreas

 

 

3 replies

Chris Hollis
Rank
Userlevel 6
Badge +15

@ak2 may I know what the TR# is?

Additionally, some things i'm sure have already been done, but i'd suggest doing again or confirming…

  1. Confirm the O/S is actually supported (may be changes between FR28 and 32), you can check the system requirements between the two releases.
     
  2. As you are seeing multiple media agents having exe crashes, it's possible AV is injecting itself into our application… is it possible to completely disable AV temporarily on one of these machines to see if that fixes the issues?

From my experience, I've seen AV, even though it's 'excluding' an app or path, still interfere with our processes causing similar symptoms to what you are sharing, so would be keen to see if this helps and we can then investigate it from there.


Let me know,

Regards,

Chris

C
Rank
Userlevel 5
Badge +14

Hello @ak2 

As Chris has mentioned a common cause of this is Antivirus causing an issue. It would be best to verify AV exclusions are in place. If they are, you can do a few things.

  1. Run a “Repair” of the software to see if that resolves the issue
  2. Run a tool like ProcMon to verify AV is not touching any of our processes or files.
  3. Check to see any evidence of System instability in Event Viewer such as low memory conditions or non-Commvault processes crashing?
  4. If the above checks out clean, in Commvault Process Manager, you can enable “Windows Error Reporting”. This will make a copy of any Commvault Process crash dumps that the OS takes. You can then create a Support Ticket and upload the dumps for review.

Antivirus Exclusions for Windows - https://documentation.commvault.com/2023e/essential/antivirus_exclusions_for_windows.html

Enabling Windows Error Reporting - https://documentation.commvault.com/2023e/expert/enabling_windows_error_reporting.html

Process Monitor v3.96 - https://learn.microsoft.com/en-us/sysinternals/downloads/procmon

 

Thank youi,

Collin

A
Rank
Userlevel 1
Badge +5

Hi Chris and Collin,
thanks for your Inputs! 

Windows Version is with CommVault 11.32 supported.
AV Tools are not installed on the Media Agents.
I ran a CommVault Instance Repair on all MediaAgents, but the App Crash Issue persistent.
I found a Crowdstrike Falcon Sensor on two of the 4 MediaAgents. However, the Agent was installed on the Media Agents before upgrade to version 11.32.

Thanks Collin for the tip about the Process Monitor!

Today I created process dumps of all affected CommVault processes and uploaded them to the CommVault ticket.

best regards,

Andreas

Reply


Terms & ConditionsCookie settings