Skip to main content
Solved

CTREE error 652 when trying to backup Nutanix distributed file share


Forum|alt.badge.img+2

Hello,

 

I am trying to set up a backup job on a Nutanix file cluster.  There are three shares.  Two are distributed shares and one is standard.  Two f the three back u without issue while the third is giving e CTREE errors

8052  216c  07/19 14:38:25 271 Servant    [---- IMMEDIATE BACKUP REQUEST ----], taskid [115] Clnt[TIDCCFG File Cluster] AppType[Windows File System][33] BkpSet[defaultBackupSet] SubClnt[Production Share] BkpLevel[Full][1]


tidcacs01 \ FileScan.log:

4788  26ec  07/19 16:25:38 271 CTREE rebuild failed with error 652 (duplicate keys rejected and listed); sysiocod=0 
4788  26ec  07/19 16:25:38 271 CTREE resumeAndRebuild: unable to rebuild index

4788  26ec  07/19 16:25:38 271 CScanTraverseFolderPrivate::~CScanTraverseFolderPrivate(130) - --- 0:00.000698
4788  26ec  07/19 16:25:38 271 CNutanixScanEngine::ProcessVolumeWorkUnit(268) - ProcessWorkUnit failed, error=0x80070306:{CNutanixScanEngine::ProcessWorkUnit(376)/Cannot perform recursive scan on this work unit due to 0x80070306:{CNutanixScanEngine::ProcessWorkUnitFull(502)} + {CvScanDB::rebuildIndex(367)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Rebuild Index Failed due to unknown error}} + {CNutanixScanEngine::ProcessWorkUnitFull(502)} + {CvScanDB::rebuildIndex(367)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Rebuild Index Failed due to unknown error}
4788  26ec  07/19 16:25:38 271 CVJobClient::setPendingCauseBackend(1068) - EventLevel=9, MessageId=0x1100008E, Flags=0x0, Event=0xE811008E:{CNutanixScanEngine::ProcessVolumeWorkUnit(271)/EventMessage.0x1100008E.17:142.(Failed to process the WorkUnit with error [One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774), Rebuild Index Failed due to unknown error (W32.774)])}
4788  26ec  07/19 16:25:38 271 CNutanixScanEngine::Process(228) - ProcessVolumeWorkUnit failed, error=0x80070306:{CNutanixScanEngine::ProcessVolumeWorkUnit(275)} + {CNutanixScanEngine::ProcessWorkUnit(376)/Cannot perform recursive scan on this work unit due to 0x80070306:{CNutanixScanEngine::ProcessWorkUnitFull(502)} + {CvScanDB::rebuildIndex(367)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Rebuild Index Failed due to unknown error}} +

Not sure what to do here.  I have Nutanix looking at logs and am wondering if it also may be something on the Commvault side of things.

Best answer by gnussbaum

Hi Mike.  We found that one of the directories was causing the issue.  It was being scanned twice, causing the job to fail.  Nutanix and Commvault developers met on a call and said it was attributed to a race condition that has been resolved.

View original
Did this answer your question?

8 replies

Forum|alt.badge.img+7
  • Vaulter
  • 62 replies
  • July 20, 2022

Please provide the following:

 

Commvault Version:

Nutanix version:


Jos Meijer
Commvault Certified Expert
Forum|alt.badge.img+17
  • Commvault Certified Expert
  • 638 replies
  • July 20, 2022

Looking at the log it seems to originate from the nutanix file system, but I'm not entirely sure.

Is this a distributed share or the standard one?

This is the initial full correct?


Forum|alt.badge.img+2
  • Author
  • Bit
  • 3 replies
  • July 21, 2022

This is a distributed share.

 

Commvault version = 11.24

Nutanix version = NCC Version: 4.3.0


Forum|alt.badge.img+7
  • Vaulter
  • 62 replies
  • July 21, 2022

@gnussbaum 

It appears that this may be a Nutanix issue. 

Based on the logs provided it appears that there was a duplicate record “error 652” found in the index. The rebuild attempt was unsuccessful possibly due to open file members, but hard to say without all the logs.  

 

 ### Attempted to rebuild and was unsuccessful ###

271 CTREE resumeAndRebuild: unable to rebuild index

 

This error below could be due to open file members that must be closed in order for the rebuild to occur or transient error.

###   (ERROR_ERRORS_ENCOUNTERED.774))-Rebuild Index Failed due to unknown error} ### 

 

Jobmanager.log may have some more details as to way this occurred. 

  • Have you opened a CV case? 
    • If so please provide the case number so we can track
  • Keep us informed on the updates from Nutanix :) 

Forum|alt.badge.img+2
  • Author
  • Bit
  • 3 replies
  • July 21, 2022

Hi,

Thanks for the update.  CV case #220719-561.  I also sent logs to Nutanix and am waiting to hear back.


Mike Struening
Vaulter
Forum|alt.badge.img+23

Hi @gnussbaum , following up on this one.  I see the case was archived without a solution.

Can you advise as to the status of the issue?

I believe this was a Nutanix issue, though hpoing you have more info.


Forum|alt.badge.img+2
  • Author
  • Bit
  • 3 replies
  • Answer
  • September 2, 2022

Hi Mike.  We found that one of the directories was causing the issue.  It was being scanned twice, causing the job to fail.  Nutanix and Commvault developers met on a call and said it was attributed to a race condition that has been resolved.


Mike Struening
Vaulter
Forum|alt.badge.img+23

Very interesting.  glad the two were able to come together and solve it for you.

I moved the Best Answer mark to your latest reply as well.

Thanks!!


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings