Skip to main content

We have an ongoing issue spread to multiple of our media agents. The problem is manifesting randomly.

Sometimes (rarely), during DDB backup we see multiple messages like this in event viewer:


63224:CVODS.exe:Failed in acquiring the lock on file [C:\Program Files\Commvault\ContentStore\Log Files\PerformanceMetrics.log] Module [PerformanceMetrics] Error[0x80070306:{CQiLogger::LockLockfile(1403)} + {CQiLogger::LockLockfile(1397)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Timed out acquiring mutex on Global\QiLogger_PerformanceMetrics}]

48044:SIDB2.exe:Failed in acquiring the lock on file [C:\Program Files\Commvault\ContentStore\Log Files\PerformanceMetrics.log] Module [PerformanceMetrics] Error[0x80070306:{CQiLogger::LockLockfile(1403)} + {CQiLogger::LockLockfile(1397)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Timed out acquiring mutex on Global\QiLogger_PerformanceMetrics}]

Sometimes DDB backup is failing during this period and it is retried and eventually completing with success, server is also unresponsive for a short period (no rdp possible), sometimes backups are going into pending state as well (clients cannot communicare with the MA) and monitoring tools cannot reach the server.

We had a commvault case open about this and it was told to us thaty there are due to network issues between the client media agent and the CS. But to be honest, this issue is only ocurring when DDB backup is running. DDB backup is scheduled to run when the backup activity is low. No lack of resources. 

My question is if anyone experienced same issues, related to the above error messages/warnings in appl win event viewer ?
 

Hello @mihaib200 

Do you see any other errors being reported in clBackup.log on the MediaAgent for the DDB Backup Job ID? Can you also check Windows Event Viewer to see if there are any VSS errors being reported?

Generally I would consider the “Failure to lock” the log files as white noise and suspect something else is happening at the same time. The OS becoming unresponsive at the same time sounds like there is something outside the Commvault Software occurring here.

During the DDB backup we ask Windows to take a VSS snapshot via API’s so I suspect a VSS issue is occurring.

Once the Server is responsive again (assuming you haven’t had to reboot) can you check the VSS through CMD and see if any errors are reported with:

  • vssadmin list writers
  • vssadmin list providers

 

Thank you,
Collin

In general, the OS level entries related to “Global\QiLogger_PerformanceMetrics” are quite commonly seen. In most cases, they do not mean anything wrong.

In your case, indeed, if the command shared by Colin do not show anything suspicious, you may run below commands in elevated prompt customizing the location of DDBs and observer the behaviour/stability of the OS


set verbose on

set context volatile

add volume D:\DDB1

add volume D:\DDB2

begin backup

create

end backup

 

If the issue is faced, possibly the DDB disks cannot cope with the load or there is an interaction with an external process… AV scanner, maybe.

Hi, we had the same Problem. Our MediaServer were not responsive für RDP about an hour.
In the EventLog nothing was related to Snapshots or anything else not usual.

It happended not every day,

We set a Blackout Windows on the Media Agent for Data Pruning during the backup of the DDB.

Since that, the error did not occur. Verify the sidb*log during Backups of the DDB.

Perhaps, this is the solution ?

Peter 

Hi, we had the same Problem. Our MediaServer were not responsive für RDP about an hour.
In the EventLog nothing was related to Snapshots or anything else not usual.

It happended not every day,

We set a Blackout Windows on the Media Agent for Data Pruning during the backup of the DDB.

Since that, the error did not occur. Verify the sidb*log during Backups of the DDB.

Perhaps, this is the solution ?

Peter 

That is also good hint!

Reply


Terms & ConditionsCookie settings