commvault.com commvault.com
Solved

Sharepoint online backup hang for one particular site

  • 17 December 2021
  • 15 replies
  • 539 views
D
Rank
Userlevel 3
Badge +13

one of SharePoint sites backup just hang in there right after complete the scan phase and stuck at 25% forever where other SharePoint subclients backup with no issue 

  • the subclient only has one site in the content
  • observed this particular subclient allocates one stream with zero application size and no throughput and no error reported 

***repeating message in the CVSPBackup.log***

 

892   1098  12/15 08:03:18 505420 CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]

892   1098  12/15 08:08:19 505420 CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]

***

I see there was a similar post for SharePoint doc backup hang issue and identified that was MS related issue by exporting via the native sharepoint powershell commands 

 

I would like to do the same test and need to find out what’s items that they tried to export - (lists and libraries only)? and what’s the command that run for the test? or any related doc

 

thank you

 

***

I found the below article and can i use this to identify if that is MS or Commvault related ?

https://morgantechspace.com/2018/06/find-and-export-all-spo-sites-using-powershell.html

icon

Best answer by Scott Reynolds 17 December 2021, 18:49

View original

15 replies

S
Rank
Userlevel 6
Badge +14

Hello @DanC 

That log entry and your description does sound like CV is waiting on the export of the site to complete to continue to parse the data. Is there anything obviously special about this one particular site? I assume you can goto the site from a webpage and it functions normally?

Also make sure you are running the backup with the option to backup only latest versions. This is on the advanced tab if you are running from java console.

One other question could this be a very large site? You see the info and size in azure SharePoint portal. 

As for your link that command looks like it's just exporting the site info not a complete data export.

D
Rank
Userlevel 3
Badge +13

Hi Scott,

actually this is an ongoing issue and never work for this particular site/subclient

backup job stuck at 25% forever, though the site functioning normal

the schedule option for this subclient with backup latest version only and the site is about hundreds GB 

 

is there anything else i can check in the Commvault or increase CV debug level?

 

dude
Rank
Badge +15

@DanC 

Have you verified the permissions under that particular site? Does the service account have the Global Administrator or the SharePoint Administrator Role over that site as well?

Can you actually “Run as” the browser to that particular site as the service account in question? does it open and can you browse through the site? 

Or you can also refer to the following article to check permissions are working as they should:

SharePoint Backup Operation Goes Into Pending State for Office 365

https://documentation.commvault.com/11.24/expert/92729_troubleshooting_article_for_office_365_with_sharepoint_01.html

 

Lastly, if the above does not give you any different results, I would indeed enable debug, maybe to 5 and increase the file size to 10MB just for troubleshooting purposes and run another backup. Will provide some additional info that can help.

 

 

 

D
Rank
Userlevel 3
Badge +13

@dude

 

SharePoint Administrator Role for the site


the scan phase with no issue :

 

4608  1684  12/12 22:01:24 505420 CVSPScan::PostScan() - scan close.
4608  1     12/12 22:01:24 505420 CVSPOffice365Client2013 CleanUp - Directory Name 
4608  1684  12/12 22:01:24 505420 CVSPScan::PostScan() - Collect files generated is less than number of streams set by user. Using total collect files value [1] as number of streams.
4608  1684  12/12 22:01:24 505420 CVSPScan::PostScan() - Running Backup in MultiStream. Number of streams = 1
4608  1684  12/12 22:01:24 505420 CVSPScan::Run() - Scan Complete in -8 seconds.

**

after scan completes, job show running but it doesn’t transfer any data and no error reported in the log. Rather repeating message in the CVSPBackup.log :

 

892   1098  12/15 08:03:18 505420 CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]

892   1098  12/15 08:08:19 505420 CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]

***

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Hi @DanC , apologies for the delayed reply (was off for the holidays).

Assuming @Scott Reynolds (who is one of our absolute top support guys) doesn’t have any other thoughts, I would suggest creating a support case for this issue.

Once you do, share the case number with me so I can track it accordingly.

https://www.linkedin.com/in/michael-struening
D
Rank
Userlevel 3
Badge +13

thanks, Mike

I’d like to rule out environmental issue first.

I will create a custom Powershell script to export a small library/files and will see how that runs and provide an update once test completes

 

 

 

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Sounds like a plan, @DanC !  Keep me posted.

https://www.linkedin.com/in/michael-struening
D
Rank
Userlevel 3
Badge +13

@Mike Struening 

Site is 4 TB 

Just wanted to give you a quick update on this


I did run script to export/download site/doc library metadata( filename, full path) via proxy and it doesn’t bring up any information and same like just hang there

 

Tried the same script for other sites which backup running fine via commvault and it shows metadata information of the site/document libraries
 

mostly like environmental issue and at this point, we’ll engage with MS to look into this further

 

I will keep you posted

 

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Appreciate the update!!!

https://www.linkedin.com/in/michael-struening
Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

@DanC , following up to see if you were able to make any progress on this one.

Thanks!

https://www.linkedin.com/in/michael-struening
D
Rank
Userlevel 3
Badge +13

@Mike Struening 

We haven’t engaged with CV support
I did additional test in my customer’s environment :

  • Exporting this particular site by using native MS Powershell command and the files being downloaded to the local temp directory that we created
  • Deleted/recreated a new subclient for this particular site and kicked off a new full backup with latest file version only
  • V1 SharePoint Online agent and went to the CV proxy server → locate UNC path which used for SP staging 
  • Waited for 20more mins and open the UNC path in the wins explorer and search for a file called “export ” , and it doesn’t bring up any search result, seeing CV created a empty folder for staging 
  • There is no error in the log, rather repeating message in every 5mins  “CvThreadPool::waitForAllThreads() - Thread Pool [SharePointBackupThreadPool]. Could not finish all work in the time limit of [300] seconds. The worker list count is [0] and busy thread count is [1] and busy thread list is [0xdac]”
  • At this point, my customer will log a case with CV support to look into this further to see why export file doesn’t being generated

I’ll update this thread with the case number once hear back from my customer , thank you

 

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Appreciate it, @DanC !  Definitely want to see this through :nerd:

https://www.linkedin.com/in/michael-struening
Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

@DanC , following up on this thread.

Was your customer able to open a support case and get a solution?

https://www.linkedin.com/in/michael-struening
D
Rank
Userlevel 3
Badge +13

@Mike Struening 

Just checked with my customer have other priority issues need to address and you might close this thread for now thanks

 

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Ok, will do.  If the situation changes, update the thread!

https://www.linkedin.com/in/michael-struening

Reply


Terms & ConditionsCookie settings