Skip to main content
commvault.com commvault.com
Solved

Failed to mount media drive

  • August 29, 2022
  • 7 replies
  • 4880 views
M
Explorer
Forum|alt.badge.img+9

Hello, Team

I am getting the error below on majority of jobs running.

I have checked the storage end and also the media agents (The LUNS are attached to the Media Agent) Everything looks good.

What could be the cause of the error.

 

 

Failed to mount the disk media in library [ARCHIVE_DISKPROD] with mount path [B:\Archive_DiskLibrary\MP10] on MediaAgent [hq_media_svr3]. Operation could not be completed in timeout interval. Please check the following: 1. Library and drive is functioning correctly. 2. Library and Drive management services are running. 3. All other MediaAgent services are running. 4. The time out period on the Expert Storage Configuration Properties Window in the CommCell Console. 5. Cleaning media in Assigned Media Group. Source: hq-vm-commserv, Process: MediaManager

Best answer by Damian Andre

Hey @Mubaraq,

I had a look at the case and the logs. Could you try restart the services on the CommServe? there is something very strange happening with the MediaManager process. 

Basically the Media Agent is sending a request to the CommServe/Media Manager process to allocate it a chunk/volume - basically a unique number that will be used to store data in for the job. Although it looks like it receives the request, it is not being processed.

In the second column of the logs is the ThreadID (2874). And what is interesting is that this thread is the only one reporting this issue. So I think perhaps Media Manager is stuck somehow, and a restart of the CommServe services may help.

 

16736 2874  08/25 22:25:45 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3068522] TIMEDOUT AFTER [653] SEC.
16736 2874  08/25 22:26:45 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3068901] TIMEDOUT AFTER [653] SEC.
16736 2874  08/25 22:31:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3071027] TIMEDOUT AFTER [649] SEC.
16736 2874  08/25 22:35:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3072386] TIMEDOUT AFTER [653] SEC.
16736 2874  08/25 22:56:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3080823] TIMEDOUT AFTER [640] SEC.
16736 2874  08/25 22:57:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3080976] TIMEDOUT AFTER [643] SEC.
16736 2874  08/25 23:04:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3083428] TIMEDOUT AFTER [639] SEC.
16736 2874  08/25 23:06:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3083831] TIMEDOUT AFTER [637] SEC.
16736 2874  08/25 23:10:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3084986] TIMEDOUT AFTER [621] SEC.
16736 2874  08/25 23:11:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3084996] TIMEDOUT AFTER [676] SEC.
16736 2874  08/25 23:27:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3092545] TIMEDOUT AFTER [634] SEC.
16736 2874  08/25 23:28:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3093908] TIMEDOUT AFTER [631] SEC.
16736 2874  08/25 23:36:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3103069] TIMEDOUT AFTER [627] SEC.
16736 2874  08/25 23:37:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3103838] TIMEDOUT AFTER [621] SEC.
16736 2874  08/25 23:41:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3105388] TIMEDOUT AFTER [630] SEC.
16736 2874  08/25 23:42:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3105622] TIMEDOUT AFTER [624] SEC.
16736 2874  08/25 23:58:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3111885] TIMEDOUT AFTER [624] SEC.
16736 2874  08/25 23:59:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3112234] TIMEDOUT AFTER [623] SEC.
 

 

 

jbb

7 replies

C
Vaulter
Forum|alt.badge.img+17

Hello @Mubaraq 

The error could mean a few different things but is especially concerning since the mount path is local storage. Essentially the OS is reporting it tried to perform an IO operation on the storage but the operation took too long and the write timed out.

 

Can you confirm the hardware is healthy? I understand it is connected and accessible but is Windows or the array hosting the LUNs reporting any errors? Also can you confirm if AV exclusions are in place?  

Recommended Antivirus Exclusions for Windows - https://documentation.commvault.com/11.23/expert/8665_recommended_antivirus_exclusions_for_windows.html

 

Thank you,
Collin

S
Apprentice
Forum|alt.badge.img+4
  • shailu89Apprentice
  • Apprentice
  • August 29, 2022

Hello @Mubaraq 

Also try to increase the mount and unmount values to see if it helps.

Right click on disk library and you can find mount and unmount timeout settings.

 

Please check media server system event viewer when this error is reported by commvault. This may give you clue on what may be wrong. 

M
Explorer
Forum|alt.badge.img+9
  • MubaraqExplorer
  • Author
  • Explorer
  • August 30, 2022

Hello, guys

So, we tried all of those and it is still same issue.

We have raised a support ticket.

Case ID is, Incident 220819-155.

Can someone help follow up, please?

jbb
Damian Andre
Vaulter
Forum|alt.badge.img+27
  • Damian AndreVaulter
  • Vaulter
  • Answer
  • August 31, 2022

Hey @Mubaraq,

I had a look at the case and the logs. Could you try restart the services on the CommServe? there is something very strange happening with the MediaManager process. 

Basically the Media Agent is sending a request to the CommServe/Media Manager process to allocate it a chunk/volume - basically a unique number that will be used to store data in for the job. Although it looks like it receives the request, it is not being processed.

In the second column of the logs is the ThreadID (2874). And what is interesting is that this thread is the only one reporting this issue. So I think perhaps Media Manager is stuck somehow, and a restart of the CommServe services may help.

 

16736 2874  08/25 22:25:45 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3068522] TIMEDOUT AFTER [653] SEC.
16736 2874  08/25 22:26:45 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3068901] TIMEDOUT AFTER [653] SEC.
16736 2874  08/25 22:31:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3071027] TIMEDOUT AFTER [649] SEC.
16736 2874  08/25 22:35:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3072386] TIMEDOUT AFTER [653] SEC.
16736 2874  08/25 22:56:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3080823] TIMEDOUT AFTER [640] SEC.
16736 2874  08/25 22:57:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3080976] TIMEDOUT AFTER [643] SEC.
16736 2874  08/25 23:04:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3083428] TIMEDOUT AFTER [639] SEC.
16736 2874  08/25 23:06:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3083831] TIMEDOUT AFTER [637] SEC.
16736 2874  08/25 23:10:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3084986] TIMEDOUT AFTER [621] SEC.
16736 2874  08/25 23:11:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3084996] TIMEDOUT AFTER [676] SEC.
16736 2874  08/25 23:27:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3092545] TIMEDOUT AFTER [634] SEC.
16736 2874  08/25 23:28:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3093908] TIMEDOUT AFTER [631] SEC.
16736 2874  08/25 23:36:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3103069] TIMEDOUT AFTER [627] SEC.
16736 2874  08/25 23:37:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3103838] TIMEDOUT AFTER [621] SEC.
16736 2874  08/25 23:41:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3105388] TIMEDOUT AFTER [630] SEC.
16736 2874  08/25 23:42:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3105622] TIMEDOUT AFTER [624] SEC.
16736 2874  08/25 23:58:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3111885] TIMEDOUT AFTER [624] SEC.
16736 2874  08/25 23:59:46 -------  SCHDLR   [        ]  *** REQUEST [REL100_MM_RESERVE_MOUNT_VOL_FOR_DRIVE_REQ] RID [3112234] TIMEDOUT AFTER [623] SEC.
 

 

 

    M
    Explorer
    Forum|alt.badge.img+9
    • MubaraqExplorer
    • Author
    • Explorer
    • September 3, 2022

    @Damian Andre thank you so much for your input.

     

    We have restarted the services on the commserve, the error persists. We also went as far as killing the jobs and restarting them.

     

    What else would you suggest, please?

    jbb
    D
    Explorer
    Forum|alt.badge.img+8
    • DaveKExplorer
    • Explorer
    • September 3, 2022

    I have seen an issue in the past where we were having a lot of  retries on an S3 LUN. This caused the media mount service to hang which stopped mount paths on different storage from mounting.

    G
    Apprentice
    Forum|alt.badge.img+4

    Hi to all of you, something similar here

    We had an issue last week when trying to upgrade from 11.24.48 to 11.28.24. I had done a vmware snapshot of the commserve before the updrade, which didnt go well and I finally did a revert and the worst happened, both Media Agents DDBs’ went corrupt. We were able to do a reconstruct, but I started to see the following message about the same time….

    Failed to mount the disk media in library [TDC1-MA03-DAS] with mount path [C:\CVDisklib\CVlib2] on MediaAgent [stdc1bckma03]. A mount path already exists at the given location. Please enter a different mount path.

    I did try to either restart the commserve services and the machine itself, same for Media Agents, no success

    Any other suggestions?

    thx in advance

     

     

    Readiverse Academy Certifications

    description of image
    Terms & ConditionsAccessibility statement