Network send failed: The network connection was lost unexpectedly
Potrei ricevere supporto per la risoluzione di questo errore?
Could I receive support in resolving this error?
Grazie
Carmine
Network send failed: The network connection was lost unexpectedly
Potrei ricevere supporto per la risoluzione di questo errore?
Could I receive support in resolving this error?
Grazie
Carmine
This error points to a network issue during the job. Does this happen every time you run a job with this client?
t has been happening for 5 days every time I try a backup of this server. Pinging the machine from commcell works. Could you please help me?
Thanks
Carmine
Hello
Could you please run the check readiness from commcell level for this client? Right Click on the client under Client and Computers > All tasks > Check readiness and see if that passes? If not, share the error message.
Secondly, login to the media you are using to backup, open commvault process manager and confirm all the services are running?
Best,
Rajiv Singal
Depending on the results from above, you might need to configure network routes
https://documentation.commvault.com/2022e/expert/7170_network_routes.html
Hello
Could you please run the check readiness from commcell level for this client? Right Click on the client under Client and Computers > All tasks > Check readiness and see if that passes? If not, share the error message.
Secondly, login to the media you are using to backup, open commvault process manager and confirm all the services are running?
Best,
Rajiv Singal
Thank you for your reply.
I had already done both the checks you suggested. Check readiness has passed and also the commvault services on the backup machine are all active. Can you give me any other suggestions?
Thanks
Carmine
Hello
Please let me know if the backup is writing now since the check readiness is passing?
If not, from the client machine, could you please share cvd and cvfwd logs?
Best,
Rajiv Singal
Hello
For the sake of completeness, I must inform you that the error is as follows:
"Network send failed: The network connection was lost unexpectedly. Source: xxxxxx process: vsbkp".
The process affected is vsbkp and for your support I am sending you the log of the vsbkp process in the hope that it will help me resolve the backup malfunction.
The machine in question is part of a cluster but the backups of the other machines in the cluster work normally.
I look forward to your feedback.
Thank you
Carmine
Hello
I am seeing crashes in the log:
4676 b40 06/30 14:50:52 122811 VSBkpDatastoreMonitor::ReportDatastore() - Datastore [DataStore] [3,158,675,816,448] bytes free [49.03%] Update Interval [300] seconds [1] VMs
4676 a78 06/30 14:50:53 122811 SdtNetLink::sendAll() - Network send failed: The network connection was lost unexpectedly.. sd=2796
4676 a78 06/30 14:50:53 122811 SdtBase::setLastErr: Setting last err [59][Network send failed: The network connection was lost unexpectedly.] RCId [311598]
4676 1a80 06/30 14:50:53 122811 SdtBase::allocateBuffer(): error_set found RCId [311598]
4676 1a80 06/30 14:50:53 122811 SdtSignature::AllocateBuffer [ERROR]Failed to allocate SDT buffer. SDT Error [Network send failed: The network connection was lost unexpectedly.]
4676 1a80 06/30 14:50:53 122811 CDeDupCache::AppendRawBuffer1() [ERROR] Failed to allocate an output buffer.
4676 1a80 06/30 14:50:53 122811 CDeDupCache::AllocateCallerBufferAndSend():[ERROR]AppendRawBuffer Failed
4676 1a80 06/30 14:50:53 122811 CDeDupCache::MoveBuffer():[ERROR]AllocateCallerBufferAndSend Failed
4676 1a80 06/30 14:50:53 122811 CDeDupCache::DispatchTagList():[ERROR]Construct-And-SendBuffer Failed
4676 1a80 06/30 14:50:53 122811 [CDeDupCache::FlushQueryQueue()] [ERROR] UpdateTagQueue-Failed
4676 1a80 06/30 14:50:53 122811 CDeDupCache::ScanAndCacheBuffer():[ERROR*] Flushing queue failed
4676 1a80 06/30 14:50:53 122811 CDeDupCache::ProcessBuffer():[ERROR]ProcessBuffer Failed
4676 1a80 06/30 14:50:53 122811 SdtSignature::ProcessBuffer - [ERROR]_pIDeDupCache::ProcessBufferAndPassToNextModule failed(13) []
This needs to be looked thoroughly. Could you please open a support case with us and share me the ticket number so that I can take it forward?
Best,
Rajiv Singal
I have collected the logs myself and looks like SIDB / bad chunks issues during backup jobids: 122811 & 122883:
SIDBEngine.log
2032 670 06/30 14:50:44 ###### 5-1-18-1 FtchBadChnks 3144 Got [1] bad chunks from MM. Last bad chunk insert time=[1688063338]. Curr Ptr [1688063338]
2032 2bc4 06/30 14:50:44 ###### 5-1-18-1 OnSubmitChunk 720 Got chunk [721030] to mark as bad
2032 2bc4 06/30 14:50:44 ###### 5-1-18-1 LeastBadChnkInsTime 158 Setting least bad chunk insert time in prop tbl [1688063338]
10948 29fc 06/30 14:50:44 ###### 5-0-17-0 FtchBadChnks 3144 Got [1] bad chunks from MM. Last bad chunk insert time=[1688063338]. Curr Ptr [1688063338]
10948 21d4 06/30 14:50:44 ###### 5-0-17-0 OnSubmitChunk 720 Got chunk [721030] to mark as bad
10948 21d4 06/30 14:50:44 ###### 5-0-17-0 LeastBadChnkInsTime 158 Setting least bad chunk insert time in prop tbl [1688063338]
7664 2514 06/30 17:30:26 ###### 5-1-18-1 ResetCounters 1960 Resetting handler counters.
4020 24d4 06/30 17:30:26 ###### 5-0-17-0 FtchBadChnks 3144 Got [1] bad chunks from MM. Last bad chunk insert time=[1688063338]. Curr Ptr [1688063338]
4020 1664 06/30 17:30:26 ###### 5-0-17-0 OnSubmitChunk 720 Got chunk [721030] to mark as bad
4020 1664 06/30 17:30:26 ###### 5-0-17-0 LeastBadChnkInsTime 158 Setting least bad chunk insert time in prop tbl [1688063338]
7664 2514 06/30 17:30:26 ###### 5-1-18-1 RefMpList 3000 Got [4] mount path list from the mediamanager.
7664 2514 06/30 17:30:26 ###### 5-1-18-1 RefMpList 3074 No disabled mount paths present.
7664 2514 06/30 17:30:26 ###### 5-1-18-1 FtchBadChnks 3144 Got [1] bad chunks from MM. Last bad chunk insert time=[1688063338]. Curr Ptr [1688063338]
7664 2a5c 06/30 17:30:26 ###### 5-1-18-1 OnSubmitChunk 720 Got chunk [721030] to mark as bad
7664 2a5c 06/30 17:30:26 ###### 5-1-18-1 LeastBadChnkInsTime 158 Setting least bad chunk insert time in prop tbl [1688063338]
Can you check the DDB and log a case for DDB issues?
Hi, I have a problem with backing up two machines in Commvault.
One is a linux machine and the other is a windows machine.
Both are part of a cluster and the other machines in the cluster back up normally without any problems.
Both machines fail the backup and give the following error: "Network send failed: The network connection was lost unexpectedly. Source: xxxxxx process: vsbkp'.
I am attaching the log of the indicated process so that the indicated malfunction can be better resolved.
Thank you
Carmine
As explained above, check DDB issues, you might need to run a DDB Verification.
https://documentation.commvault.com/11.28/expert/100399_data_verification_of_deduplicated_data.html
Then log a case if issue persists.
Could you kindly tell me where to find and perform the DDB check. In Commandcenter or in Console? Could you explain to me how to proceed and what steps to follow? I would be infinitely grateful if you could give me this information.
Thank you
That’s the link I have provided above.
Did you check in our Documentation:
Hi Sebastien,
I ran the full DDB check as requested and found no errors. I have restarted the backups, but they always stop in pending with error 82:127 “Network send failed: The network connection was lost unexpectedly. Source: xxxxxx process: vsbkp".
I would be very grateful if you could advise me on how to solve this problem.
Thanks in advance.
Carmine
Hi
Ok, perfect, can you delete or rename that folder on the VSA, then try a new Backup:
C:\Program Files\Commvault\ContentStore\Base\Temp\vmware-SYSTEM
Best Regards,
Sebastien
Incremental will do.
Hi
I have followed your suggested procedure. I deleted the indicated folder and restarted the incremental backup of the computer.
The result has not changed. The error is still the same.
Please help me.
Thank you.
Carmine
Do you only have 1 VSA?
Are you able to deploy another VSA?
Otherwise, just log a case with Support and we will check on a session.
I have already opened a ticket for this problem:
Incident 230703-404
The customer has not had a backup since 13 June. I urgently need the resolution.
Can you call me at +390473491313 ?
So you connect directly to the machines?
Thank you
Carmine
My colleague checked but like we said there is an issue with SIDBEngine and he needs the logs from the Media Agent _CL-SAP.
So now we have them he will check and get back to you.
Correct the best is to send all the logs, CS, MA and VSA.
I will ask the engineer to add a key the Media Agents and run a new Backup again:
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.