Skip to main content
Question

Disk consolidation process causing server application timeout


Forum|alt.badge.img+3

Hi Team,

We are experiencing an issue with one of the servers, which disconnects from the application at 5 AM EST every day. Upon investigation, we found that a snapshot removal operation or disk consolidation of the backup snapshot occurs during this time.

We have a scheduled backup at 5 PM EST every day during non-business hours. However, Commvault triggers the disk consolidation operation at 4:58 AM or 5:00 AM EST every morning, causing an outage on the server during this time.

Is it possible to change the disk consolidation window/schedule/pattern?

4 replies

Forum|alt.badge.img+7

Hi ​@Kumar Devaraj ,

Good day!

During the backup process, Commvault (CV) creates a snapshot for the VM. At the end of the backup job, a request is made to delete the snapshot. If the snapshot removal doesn't complete in time, the job will be marked as complete, and the snapshot will be added to the cleanup queue.

After the initial snapshot removal request via the VDDK API is sent, if it fails, subsequent requests are made with exponential backoff delays: 4 minutes, then 9 minutes, 16 minutes, 25 minutes, and so on, until 4 hours have passed. After 4 hours, further requests will continue. These delays provide sufficient time for the host to complete the task.

If you observe that consolidation requests are being sent even after the backup is complete, please check why the initial snapshot removal request isn't completing in a timely manner on the VMware side. If the snapshot is properly removed, CV will not send further requests.

If snapshots are left unattended for too long, they will continue to grow dynamically, consuming storage space until the datastore reaches capacity. Additionally, prolonged snapshot retention can degrade virtual machine performance. To mitigate these impacts and maintain the stability of the environment, Commvault’s cleanup mechanism operates continuously and cannot be modified, ensuring that snapshots are removed promptly to prevent storage and performance issues.

 

Regards,

Sureshkumar S

 

 


Forum|alt.badge.img+3

Hi Suresh,

Thank you for the update!

We have checked the Vcenter logs/events for the backup process. The backup job is triggered at the scheduled time after 5 PM EST, and the snapshot creation and deletion were completed immediately after.

I am attaching screenshots with timestamps. Please check and let us know why CommVault is attempting disk consolidation again when the snapshot removal was already completed successfully.

 

  1. scheduled backup started
  2. Backup snapshot creation completed

  3. Snapshot removal completed

  4. disk consolidation is complete here at 5.44PM

    Next day morning again snapshot creation happening 

    during this time application outage occurred
     
    snapshot deletion happened again here

    why CommVault attempting disk consolidation again in the morning?

Users facing outage during morning when snap creation and deletion happened again. 

 

 


Forum|alt.badge.img+7

Hi ​@Kumar Devaraj ,

Kindly raise a support case to review this

Get all logs from CV/VMware for review

Regards,

Sureshkumar S


Forum|alt.badge.img+3

Hi ​@Sureshkumar S,

Sure, I will raise support case and I will update here.

Thanks for your help.

 

Regards,

Kumar D


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings