We have an ongoing issue spread to multiple of our media agents. The problem is manifesting randomly.
Sometimes (rarely), during DDB backup we see multiple messages like this in event viewer:
63224:CVODS.exe:Failed in acquiring the lock on file [C:\Program Files\Commvault\ContentStore\Log Files\PerformanceMetrics.log] Module [PerformanceMetrics] Error[0x80070306:{CQiLogger::LockLockfile(1403)} + {CQiLogger::LockLockfile(1397)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Timed out acquiring mutex on Global\QiLogger_PerformanceMetrics}]
48044:SIDB2.exe:Failed in acquiring the lock on file [C:\Program Files\Commvault\ContentStore\Log Files\PerformanceMetrics.log] Module [PerformanceMetrics] Error[0x80070306:{CQiLogger::LockLockfile(1403)} + {CQiLogger::LockLockfile(1397)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Timed out acquiring mutex on Global\QiLogger_PerformanceMetrics}]
Sometimes DDB backup is failing during this period and it is retried and eventually completing with success, server is also unresponsive for a short period (no rdp possible), sometimes backups are going into pending state as well (clients cannot communicare with the MA) and monitoring tools cannot reach the server.
We had a commvault case open about this and it was told to us thaty there are due to network issues between the client media agent and the CS. But to be honest, this issue is only ocurring when DDB backup is running. DDB backup is scheduled to run when the backup activity is low. No lack of resources.
My question is if anyone experienced same issues, related to the above error messages/warnings in appl win event viewer ?