Skip to main content
Question

VM backup performance troubleshooting

  • September 7, 2023
  • 3 replies
  • 362 views

Forum|alt.badge.img+5

Hi,

 

I am trying to find the bottle net of a VM backup.

When I looked at the CVPerfMgr.log in the MA, I see that below disk read takes lots of time compared to the rest.

|*49925915*|*Perf*|19096983| Virtual Server Agent

…...

|*49925915*|*Perf*|19096983|      |_Disk Read........................................................................................      3827              120142692702  [111.89 GB] [105.25 GBPH]

 

Also Reader/Writer Pipeline Modules wait for the about the same time for data.

 

|*49925915*|*Perf*|19096983| Reader Pipeline Modules[Client]
|*49925915*|*Perf*|19096983|  |_CVA Wait to received data from reader................................................................      3820

+

|*49925915*|*Perf*|19096983| Writer Pipeline Modules[MediaAgent]
|*49925915*|*Perf*|19096983|  |_SDT-Tail: Wait to receive data from source...........................................................      3907                 285051275  [271.85 MB]  [Samples - 470572] [Avg - 0.008303] [0.24 GBPH]

 

With above, I am guessing the issue is with the Virtual Server Agent/ Disk reading.

But when I looked that the VM/ ESX performance, I don't see any resource maxed out during the backup time.

 

Does anyone know what components are included in this Virtual Server Agent entry in the log? Is it just the local VM/ESX performance or it is network related too?

 

Kind regards,

Boyi

 

3 replies

Rajiv
Vaulter
Forum|alt.badge.img+12
  • Vaulter
  • 323 replies
  • September 7, 2023

Hello @Boyi 

Please let me know the following: 

  1. What is the commserve version and vddk version?
  2. How many VM’s are impacted? 
  3. What is the transport mode being used?
  4. If you double click on a job, which component out of (read, network, write) is taking the most time?
  5. Can you share and check vsbkp.log on the access node and check for stat id and look for stats where exactly is the bottleneck.
  6. Also, you can watch this informative video and can run a test against the datastore: 

Best,

Rajiv Singal


Forum|alt.badge.img+5
  • Author
  • Byte
  • 8 replies
  • September 12, 2023

Hi Rajiv,

 

What is the commserve version and vddk version? Commserve is at 11.24.112. I don't have access to customer’s vCentre/ESX so I dont know about vddk.
How many VM’s are impacted? this is a new setup so we only tested the backup on one VM. But there are other VMs in the same/different ESXs that are within the same vCentre.
What is the transport mode being used? NBD
If you double click on a job, which component out of (read, network, write) is taking the most time? Read and DDB.
Can you share and check vsbkp.log on the access node and check for stat id and look for stats where exactly is the bottleneck. I looked into vsbkp.log and cannot find info for bottlenet. Can you give me an example? In the CVPerfMgr.log I see that it is the disk read as I posted originally.


Rajiv
Vaulter
Forum|alt.badge.img+12
  • Vaulter
  • 323 replies
  • September 12, 2023

Hello @Boyi 

If disk is taking time to read its probably a datastore issue, for example higher IOPS. Were you able to run the test as mentioned in the video I shared above? DDB can take time if its a fresh/first backup.

In vsbkp.log you should see something with stat-id and check the disk performance there. Also, you can find the VDDK version in the vsbkp.log itself.  If you can share the log here as an attachment, I can check and share the result with you here. 

If you would need proper assistance in getting this rectified and resolved, you can raise a case with us and we can assist you further.

Best,

Rajiv Singal


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings