Sharepoint online backup hang for one particular site

Hi Scott,

actually this is an ongoing issue and never work for this particular site/subclient

backup job stuck at 25% forever, though the site functioning normal

the schedule option for this subclient with backup latest version only and the site is about hundreds GB 

is there anything else i can check in the Commvault or increase CV debug level?

@DanC 

Have you verified the permissions under that particular site? Does the service account have the Global Administrator or the SharePoint Administrator Role over that site as well?

Can you actually “Run as” the browser to that particular site as the service account in question? does it open and can you browse through the site? 

Or you can also refer to the following article to check permissions are working as they should:

SharePoint Backup Operation Goes Into Pending State for Office 365

https://documentation.commvault.com/11.24/expert/92729_troubleshooting_article_for_office_365_with_sharepoint_01.html

Lastly, if the above does not give you any different results, I would indeed enable debug, maybe to 5 and increase the file size to 10MB just for troubleshooting purposes and run another backup. Will provide some additional info that can help.

@dude

SharePoint Administrator Role for the site

the scan phase with no issue :

4608  1684  12/12 22:01:24 505420 CVSPScan::PostScan() - scan close.
4608  1     12/12 22:01:24 505420 CVSPOffice365Client2013 CleanUp - Directory Name 
4608  1684  12/12 22:01:24 505420 CVSPScan::PostScan() - Collect files generated is less than number of streams set by user. Using total collect files value [1] as number of streams.
4608  1684  12/12 22:01:24 505420 CVSPScan::PostScan() - Running Backup in MultiStream. Number of streams = 1
4608  1684  12/12 22:01:24 505420 CVSPScan::Run() - Scan Complete in -8 seconds.

**

after scan completes, job show running but it doesn’t transfer any data and no error reported in the log. Rather repeating message in the CVSPBackup.log :

892   1098  12/15 08:03:18 505420 CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]

892   1098  12/15 08:08:19 505420 CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]

***

Hi @DanC , apologies for the delayed reply (was off for the holidays).

Assuming @Scott Reynolds (who is one of our absolute top support guys) doesn’t have any other thoughts, I would suggest creating a support case for this issue.

Once you do, share the case number with me so I can track it accordingly.

thanks, Mike

I’d like to rule out environmental issue first.

I will create a custom Powershell script to export a small library/files and will see how that runs and provide an update once test completes

 

Sounds like a plan, @DanC !  Keep me posted.

@Mike Struening 

Site is 4 TB 

Just wanted to give you a quick update on this

I did run script to export/download site/doc library metadata( filename, full path) via proxy and it doesn’t bring up any information and same like just hang there

Tried the same script for other sites which backup running fine via commvault and it shows metadata information of the site/document libraries
 

mostly like environmental issue and at this point, we’ll engage with MS to look into this further

I will keep you posted

Appreciate the update!!!

@DanC , following up to see if you were able to make any progress on this one.

Thanks!

@Mike Struening 

We haven’t engaged with CV support
I did additional test in my customer’s environment :

• Exporting this particular site by using native MS Powershell command and the files being downloaded to the local temp directory that we created
• Deleted/recreated a new subclient for this particular site and kicked off a new full backup with latest file version only
• V1 SharePoint Online agent and went to the CV proxy server → locate UNC path which used for SP staging 
• Waited for 20more mins and open the UNC path in the wins explorer and search for a file called “export ” , and it doesn’t bring up any search result, seeing CV created a empty folder for staging 
• There is no error in the log, rather repeating message in every 5mins  “CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]”
• At this point, my customer will log a case with CV support to look into this further to see why export file doesn’t being generated

I’ll update this thread with the case number once hear back from my customer , thank you

Appreciate it, @DanC !  Definitely want to see this through 

@DanC , following up on this thread.

Was your customer able to open a support case and get a solution?

@Mike Struening 

Just checked with my customer have other priority issues need to address and you might close this thread for now thanks

Ok, will do.  If the situation changes, update the thread!