Skip to main content
commvault.com commvault.com
Solved

OneDrive user backup with large amount of data issue

  • October 12, 2021
  • 6 replies
  • 804 views
K
Byte

Hello Commvault Community, 

 

Today I come with the question of what might be causing the problem with the OneDrive backup job, but only for one user who has a large amount of data.

 

Users with little data are successfully backed up. 

 

This user was separated from a separate Sub-Client and a backup job was forced. It runs long and loops.

In the log on the Access Node (CvCloudBkpOneDrive.log) you can see for this backup job:

 

1) For a long time, for different files with different file extensions
 

14220 38b0 10/07 15:56:06 222703 OneDriveBackup :: RecordFailedItem (2090) - * Backup failed for (/OneDrive/xyzxyzxyzxyzxyzxyzxyzxyzxyzxyzxyz
14220 38b0 10/07 15:56:06 222703 OneDriveBackup :: RecordFailedItem (2091) - * [File download failure.]
14220 38b0 10/07 15:56:06 222703 OneDriveBackup :: RecordFailedItem (2090) - * Backup failed for (/OneDrive/xyzxyzxyzxyzxyzxyzxyzxyzxyzxyzxyzxyzxyz

 

2) After some time you can see an entry about the problem with Throttling:

4220 1 10/07 16:16:29 222703 ### callbackPermissionList - Throttling error, responseId (10524)
14220 1 10/07 16:16:29 222703 ### callbackPermissionList - Throttling error, responseId (10523)
14220 1 10/07 16:16:29 222703 ### - - * Wait (120) seconds because of throttling error in batch request
14220 1 10/07 16:18:59 222703 ### callbackPermissionList - Throttling error, responseId (11006)
14220 1 10/07 16:18:59 222703 ### callbackPermissionList - Throttling error, responseId (11010)
14220 1 10/07 16:18:59 222703 ### callbackPermissionList - Throttling error, responseId (11009)
14220 1 10/07 16:18:59 222703 ### callbackPermissionList - Throttling error, responseId (11008)
14220 1 10/07 16:18:59 222703 ### - - * Wait (120) seconds because of throttling error in batch request
14220 38b0 07/10 16:24:37 222703 OneDriveBackup :: StartBackup (2705) - * Start Backup for folders. mMapFileMetadata [14140]
14220 38b0 10/07 16:24:37 222703 OneDriveBackup :: UpdateProgress (1268) - * progress = 0.007072 (1/14140)
14220 1 10/07 16:24:37 222703 ### BatchGetPermissions - * Total elapsed Time for 14140 files: 0 Days 0 Hours 27 Minutes 30 Seconds 127 Milliseconds
14220 38b0 10/07 16:24:41 222703 OneDriveBackup :: StartBackup (2761) - * Download start for 13043 items
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - result: Amib.Threading.Internal.WorkItem + WorkItemResult
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - downloadsize: 6235919, requestindex: 4
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - result: Amib.Threading.Internal.WorkItem + WorkItemResult
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - downloadsize: 14312, requestindex: 5
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - result: Amib.Threading.Internal.WorkItem + WorkItemResult
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - downloadsize: 3260636, requestindex: 1
14220 1 10/07 16:25:06 222703 ### callbackDownloadForFileInfo_afterdownload - result: Amib.Threading.Internal.WorkItem + WorkItemResult

 

My question is, what could be the reason for the behavior that for small users OneDrive backup works fine, and for this one large user the backup job is not working.

 

When a job is stopped at Job Events, it indicates "Backup job (222703) has been converted to a full.

 

I started looking, and recommended to check if by any chance the situation will improve after setting up Network Throttling on the subclient level, maybe you will have an idea where to look for a solution.’

 

Thanks for help

Regards
Kamil

Best answer by Scott Reynolds

@Kamil 

From the log cuts you posted it looks like it's possible this particular user has not only a large amount of data but possibly some very large files as well? The cuts are showing we are attempting to break up with large files into download chunks. The second section just shows we are waiting for the download to complete so the job stops reporting progress while this large download is occurring.

I would start by creating a few more azure apps to help with throttling. Another 5 for a total of 10 should be sufficient. Once that is complete run a backup of just this particular user without when the other backup is not running and see how it performs.

If you still have an issue open a support case and respond with the case number. It is possible we can provide a setting to skip the large files to see if the backup completes without them depending on the log review.

View original
Did this answer your question?

6 replies

Mike Struening
Vaulter
Forum|alt.badge.img+23

Hey @Kamil , assuming this is OneDrive for Office 365, can you confirm how many Azure Apps you have configured?  You might be getting throttled on the Azure side.

Multiple Apps allow us to move to different apps as throttling gets close to occurring.

https://www.linkedin.com/in/michael-struening
MichaelCapon
1 person likes this
  • MichaelCapon
    MichaelCapon
K
Byte
  • Kamil
  • Author
  • 143 replies
  • October 13, 2021

Hello @Mike Struening 

 

Thanks for quick reply. 

 

Customer have the minimum amount and is given in the documentation, i.e. 5 (azureapps.png)

 

Regards,

Kamil

Mike Struening
Vaulter
Forum|alt.badge.img+23

@Kamil , how many users do you have total?  And how big are these larger ones?

I’m reaching out to some of our internal support team members to see what they can suggest.

https://www.linkedin.com/in/michael-struening
S
Vaulter
Forum|alt.badge.img+14

@Kamil 

From the log cuts you posted it looks like it's possible this particular user has not only a large amount of data but possibly some very large files as well? The cuts are showing we are attempting to break up with large files into download chunks. The second section just shows we are waiting for the download to complete so the job stops reporting progress while this large download is occurring.

I would start by creating a few more azure apps to help with throttling. Another 5 for a total of 10 should be sufficient. Once that is complete run a backup of just this particular user without when the other backup is not running and see how it performs.

If you still have an issue open a support case and respond with the case number. It is possible we can provide a setting to skip the large files to see if the backup completes without them depending on the log review.

neuwiesener
Byte
Forum|alt.badge.img+7
  • neuwiesener
  • Byte
  • 38 replies
  • December 27, 2021

@Mike Struening  Can someone please explain what these stats stand for:

SdtTail :: logStats: Clnt [srv-an1], PId [13508], Cnt - BsyProcsg / Allocs [121/77080]; Time - Procsg / NwRecv / Total [220111/7051/227162] secs, Bytes recvd [38459688] RCId [316520]

I am guessing: this stats line has three parts Count, time and bytes concering a client? 

Cnt - BsyProcsg / Allocs [121/77080] 

Cnt = Count? 

BsyProcsg = Busy Processing ? 

Allocs = Time / count of buffers? allocated? 

SdtTail = SDT Simpana data transfer? Tail of the datapipe?

Bytes recvd [38459688] = bytes received

 

Time - Procsg / NwRecv / Total = 

Procsg = Process signature?? 

 

 

Regards, Ashutosh
Mike Struening
Vaulter
Forum|alt.badge.img+23

Hi @neuwiesener , hope your holiday was restful!

Clnt is the Client name

PId is the process ID

Count is (I believe) the count of transfers

SdtTail is the transfer pipeline (it’s either simple or simpana)

I’ll check with some of our Media Management folks to fill in what I’m missing.

https://www.linkedin.com/in/michael-struening

Reply


Most helpful members this week

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings