Question

HPE Catalyst VMware backups [62:2910] [-1404 OSCLT_ERR_SERVER_OFFLINE]

  • 24 November 2023
  • 1 reply
  • 155 views

I recently configured Catalyst over ethernet and I haven’t any issues during the integration, even a single VM backup test was successful, however when I scheduled a subclient with multiple VMs (5) the job is going to waiting state with the below error:

 

Error Code: [62:2910]
Description: Error occurred in Disk Media, Path [CATVMsManagement\1NDD43_11.24.2023_02.03\CV_MAGNETIC\V_3] [-1404 OSCLT_ERR_SERVER_OFFLINE]. For more help, please call your vendor's support

 

I did validate the Store in Commvault and it is in status Ready and access Read/Write, Moun Path Allocation Policy is set to Maximum Allowed Writers. Also ran a mountpath validation on the store and it was successful. The error is only thrown when there is more than 1 VM in the subclient.

 

Is there something else to configure to allow multiple data streams run on the StoreOnce side?

 

Here a sniplet of  Job Manager.log

19296 66c8  11/23 22:31:12 19  Servant    Remote Stream Allocation request for [4] streams of type [1] received. Request Identifier [Global\19_564_1700911884_WAIT$Global\19_564_1700916793_SHM]. Client [vesb2bagentcomv]
19296 6440 11/23 22:31:20 19 Resource Alloc Obj Sending resource allocation response. Successfully allocated [4] of [4] streams. Request Identifier [Global\19_564_1700911884_WAIT$Global\19_564_1700916793_SHM].
19296 66c8 11/23 22:32:51 19 Scheduler Set pending cause [Error occurred in Disk Media, Path [CATVMsManagement\1NDD43_11.24.2023_02.03\CV_MAGNETIC\V_3] [-1404 OSCLT_ERR_SERVER_OFFLINE]. For more help, please call your vendor's support hotline.]::Client [vesb2bagentcomv] Application [cvd] Message Id [671088705] Error code [40:65] RCID [0] ReservationId [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[1040190302].
19296 60b4 11/23 22:32:51 19 Scheduler Ignored pending cause [[Media write operation failure] [HP-StoreOnce media] [Error writing to the media.]]::Client [vesb2bagentcomv] Application [cvd] Message Id [218104052] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
19296 60b4 11/23 22:33:26 19 Scheduler Ignored pending cause [Error occurred in Disk Media, Path [CATVMsManagement\1NDD43_11.24.2023_02.03\CV_MAGNETIC\V_4] [-1404 OSCLT_ERR_SERVER_OFFLINE]. For more help, please call your vendor's support hotline.]::Client [vesb2bagentcomv] Application [cvd] Message Id [671088705] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[1040190302].
19296 66c8 11/23 22:33:26 19 Scheduler Ignored pending cause [[Media write operation failure] [HP-StoreOnce media] [Error writing to the media.]]::Client [vesb2bagentcomv] Application [cvd] Message Id [218104052] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
19296 66c8 11/23 22:34:16 19 Servant Remote free streams request received for rcid [15]
19296 60b4 11/23 22:34:31 19 Servant Remote free streams request received for rcid [16]
19296 66c8 11/23 22:35:04 19 Scheduler Ignored pending cause [Error occurred in Disk Media, Path [CATVMsManagement\1NDD43_11.24.2023_02.03\CV_MAGNETIC\V_7] [-1404 OSCLT_ERR_SERVER_OFFLINE]. For more help, please call your vendor's support hotline.]::Client [vesb2bagentcomv] Application [cvd] Message Id [671088705] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[1040190302].
19296 60b4 11/23 22:35:04 19 Scheduler Ignored pending cause [[Media write operation failure] [HP-StoreOnce media] [Error writing to the media.]]::Client [vesb2bagentcomv] Application [cvd] Message Id [218104052] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
19296 60b4 11/23 22:35:31 19 Servant Remote Stream Allocation request for [2] streams of type [1] received. Request Identifier [Global\19_564_1700904951_WAIT$Global\19_564_1700898611_SHM]. Client [vesb2bagentcomv]
19296 6440 11/23 22:35:50 19 Resource Alloc Obj Sending resource allocation response. Successfully allocated [2] of [2] streams. Request Identifier [Global\19_564_1700904951_WAIT$Global\19_564_1700898611_SHM].
19296 60b4 11/23 22:36:43 19 Scheduler Ignored pending cause [The other side has closed the network connection gracefully]::Client [vesb2bagentcomv] Application [vsbkp] Message Id [1375731854] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
19296 66c8 11/23 22:36:43 19 Servant Remote free streams request received for rcid [12]
19296 60b4 11/23 22:36:43 19 Scheduler Ignored pending cause [The other side has closed the network connection gracefully]::Client [vesb2bagentcomv] Application [vsbkp] Message Id [1375731854] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
19296 66c8 11/23 22:36:43 19 Servant Remote free streams request received for rcid [14]
19296 66c8 11/23 22:36:46 19 Servant Remote free streams request received for rcid [19]
19296 66c8 11/23 22:36:48 19 Servant Remote free streams request received for rcid [20]
19296 60b4 11/23 22:36:48 19 Scheduler Ignored pending cause [The other side has closed the network connection gracefully]::Client [vesb2bagentcomv] Application [vsbkp] Message Id [1375731854] RCID [0] Reservation Id [0]. Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
19296 66c8 11/23 22:36:48 19 Servant Remote free streams request received for rcid [17]
19296 60b4 11/23 22:36:48 19 Scheduler Phase [Failed] message received from [VESB2BAGENTCOMVLTV01] Module [vsbkp] Token [19:4:1:0:0:1714] restartPhase [0]
19296 2e3c 11/23 22:36:48 19 Operations Invalidate Incomplete Vm Pseudo Jobs.
19296 60b4 11/23 22:36:48 19 JobSvr Obj Phase [4-Backup] for Backup Job Failed. Backup will continue with phase [Backup].
19296 60b4 11/23 22:36:48 19 ArchiveMgr JMArMgr::releaseAllJobStreams() Calling CvResourceManager::releaseJobStreams(jobType=1,appType=106,logicalRelease=true, retainStreamsForIndex=false)
19296 2e3c 11/23 22:36:48 20 JobSvr Obj phase-ready-for-commit context flag has been reset for the job because there is no closed chunks.
19296 2e3c 11/23 22:36:48 21 JobSvr Obj phase-ready-for-commit context flag has been reset for the job because there is no closed chunks.
19296 2e3c 11/23 22:36:48 22 JobSvr Obj phase-ready-for-commit context flag has been reset for the job because there is no closed chunks.
19296 66c8 11/23 22:36:48 19 Scheduler Got message to set job state to 3 but the job is not running.

 


1 reply

Userlevel 6
Badge +15

@imagana 

Mind confirming for me what feature release / maintenance release your environment is on presently?

I’ll do some digging for you once I’ve confirmed.

Regards,

Chris 

Reply