Skip to main content

Hello All,

 

I am facing the same issue.  configured the file system backup for the server and after starting its full backup getting the error “encountered failure in receiving data aThe SDT data transfer was terminated on a request from the Job Manager.]”

Applied the above suggested media management setting but no luck. 

 

17708 2e14  08/11 13:47:45 4370936 CPipelayer::SendPipelineBuffer() - Tail has reported error e94]rThe SDT data transfer was terminated on a request from the Job Manager.]. Cannot continue.

17708 2e14  08/11 13:47:45 4370936 5PIPELAYER  ] Error in flushing the current buffer.

17708 2e14  08/11 13:47:45 4370936 CVArchive::WriteBuffer() - Cannot send the buffer. Ret b268435460]

17708 2e14  08/11 13:47:45 4370936 CFileBackup::WriteBuffer(1683) - writeBuffer failed

17708 2e14  08/11 13:47:45 4370936 CFileBackup::HandleReadAndSendFileDataError(1497) - WriteBuffer failed

17708 2e14  08/11 13:47:45 4370936 CBackupBase::DoBackup(3689) - ReadAndSendFileData indicates FAIL_BACKUP

17708 2e14  08/11 13:47:45 4370936 CBackupBase::DoBackup(2608) - --- 8:29.499975 

17708 2e14  08/11 13:47:45 4370936 FsBackupTw::Run(510) - --- 8:29.502717 ObjectId=1, CollectFileName=C:\Program Files\Commvault\ContentStore\iDataAgent\JobResults\CV_JobResults\iDataAgent\FileSystemAgent\2\3585\NumColTot1.cvf

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::Run_WaitForThreadsToComplete(5364) - Wait for multiple objects signalled. Releasing one reservation used of object i1]

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::Run_WaitForThreadsToComplete_ReleaseStream(5412) - Worker thread reported error while processing collect file eC:\Program Files\Commvault\ContentStore\iDataAgent\JobResults\CV_JobResults\iDataAgent\FileSystemAgent\2\3585\NumColTot1.cvf]

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::PutInFileTheFailedFilesList(5662) - FailedFileCnt 60]

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::MarkCFAsProcessedInSubStoresReturned(10527) - mapSubStoreByCollect - size S0]

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::Run_WaitForThreadsToComplete_ReleaseStream(5496) - iLatestObjectSignalled=i1], iCurrentlyActiveStreams=C0]

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::Run_WaitForThreadsToComplete(5351) - --- 8:29.502192 

17708 5974  08/11 13:47:45 4370936 FsBackupCtlr::Run(3588) - iLatestObjectSignalled= 1], iMaxNumberOfThreads==1], iCurrentlyAvailableStreams=C1], iCurrentlyActiveStreams=,0]

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4352) - +++ 

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4420) - Updating counts, FailedFiles=0, FailedFolders=0

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4480) - Leaving archive file marked invalid

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4485) - Closing pipeline

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4491) - Sending PL_LAST to tail.

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4495) - 74370936] backupState(1=good)=0, closePlArgs.backupState=0

17708 5974  08/11 13:47:45 4370936 CVArchive::ClosePipeline() - Resetting allocated stream

17708 5974  08/11 13:47:45 4370936 7PIPELAYER  ] Data Pipe Is Down

17708 5974  08/11 13:47:45 4370936 CCVAPipelayer::SendPerfCountersToDSBackup() - Failed to allocate Perf-counters command buffer

17708 5974  08/11 13:47:45 4370936 CCVAPipelayer::ClosePipeline() - About to destroy Data Mover

17708 5974  08/11 13:47:45 4370936 PIPELAYER  ] Data Pipe Is Down

17708 5974  08/11 13:47:45 4370936 CCVAPipelayer::SendCommandToDSBackup() - Failed to allocate a command buffer

17708 5974  08/11 13:47:45 4370936 CCVAPipelayer::ClosePipeline() - Failure sending datamover destroy

17708 5974  08/11 13:47:45 4370936 CPipelayer::ShutdownPipeline() - stat- SDT C00000148929A00C0] nduration - 513 seconds]

17708 5974  08/11 13:47:45 4370936 CPipelayer::ShutdownPipeline() - pipeline has already been shutdown

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4503) - Backup: Errors in closing the pipeline.

17708 5974  08/11 13:47:45 4370936 CBackupBase::Close(4505) - Back from closePipeline

Hi @G@rim@ !  I moved this to its own thread so we can better solve your issue (and the other thread didn’t resolve it for you).

Moved from:

Can you answer a few questions?

  1. What version is the Commserve as well as this client?  I have seen some issues where a mismatch causes this issue.
  2. Is there a firewall between the CS and Client? 
    1. Is the Media Agent the same as the CS?  If not, is there a firewall involved?

The error indicates that a connection was dropped and is usually indicative of a firewall issue or network issue.

We often notice setting up a dedicated consistent firewall within Commvault alleviates many network issues:

https://documentation.commvault.com/commvault/v11/article?p=7198.htm

@MichaelCapon I have checked the Job controller logs and i can see below errors in it. 

 

912  1d40  08/25 00:51:54 4407044 Control    JMASYNC sendControlMessageAsync network error while sending control message. Handle 81539], TimeOutTfalse],ErrorString S0x80070306:{ServiceCallBack(1241)/Couldn't connect to remote service. Handle (81539), Message (0x80070306:{ServiceCallBack(1210)} + {CCvIOCPServer::WorkerThread(1939)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Connection closed.}), connection 

 

 

@Mike Struening Both Client and Commserve are running on the same version. and no firewall exist between client and MA. 

Also i have tried this setting but doesn’t worked for us. 

@G@rim@ , can you monitor the traffic between the 2 servers (with Wireshark or similar) to see if there are any drops?

The logging indicates network drops.  the One Way Firewall config helps by keeping a persistent pipe but if that is not working, it’s likely something deeper.

I would have expected the one-way persistent tunnel to resolve that error from occurring. - This error is usually caused by connection drop.
We may need to check clBackup, cvd and cvfwd logs (On the Client) leading up to the time of the error. 

 

Best Regards,

Michael

Reply


Terms & ConditionsCookie settings