Skip to main content
commvault.com commvault.com
Solved

DDB and Index Cache restore failing

  • January 25, 2022
  • 4 replies
  • 1190 views
Z
Bit
Forum|alt.badge.img+2

Hi

 

Good day

 

So there was a power outage, and I lost the SAN LUNs I provisioned for both DDB, Index Cache and even the storage repository. They were mapped/zoned to the Media Agent (Server) initially.

 

Unfortunately, the guy on ground provisioned new storage for these DDBs AND Index Cache, changed the drive letters and reassigned the partitions

 

I have brought back the LUNs and the backup files within are still intact.

 

However, the location specified in the media library is offline.

 

I can’t restore anything from this location.

Please I need help, because we need to restore a file server

Best answer by Laurent

1st, as suggested by Mike, call the support and they’ll be able to assist you. That’s an emergency and they’re handful for such.

 

DDB is only used to backup. 

From what I understood, you just need to restore data, not to backup.

So, of course, you’ll get tons of warning/critical errors from the Commserve/MA logs reporting a ddb or devices are down, but this should not prevent your disk library to be back online, if it had been brought up exactly as it was before the issue.

The devil lies in the details, and maybe here in your MA where you have you disklibrary, the LUN IDs could have changed, the windows disk IDs (SAN mostly equals to multipathing so after a hardware failure paths could have changed also), resulting in data inside the volume is OK, but from a topdown hardware volume point of view, the device hosting this volume(s) for Commvault is not exactly the same. Then Commvault reports that disk library as offline.

In this disk library, do you have more than 1 mount path ?

What is the status of the mount path : is it offline ? Enabled ? Compare this and details to the physical volume inside the OS.

And also make sure for this MA that all backup, pruning, data aging activities are disabled until situation is solved and you recovered your data.

 

View original
Did this answer your question?
If you have a question or comment, please create a topic

4 replies

Mike Struening
Vaulter
Forum|alt.badge.img+23

To confirm, when you say you can’t restore anything from this location, are you saying the restore is failing now (because it is not connected to the drives properly), or you can’t restore the actual source data at all?

The crux of the issue is what data you can pull of the downed drives.  If the data is there, can you return the LUNs back to their proper drives?

It sounds like (and confirm) that you have new hardware connected, but with new drive letters (and they contain no data).

You should be able to restore the data and fix the drive letters and everything should (in theory) work.

However, before we can determine the best way forward, we need to know what things look like now, and what you can and can’t get the data from.

https://www.linkedin.com/in/michael-struening
Z
1 person likes this
  • Z
    zadok001
Z
Bit
Forum|alt.badge.img+2
  • zadok001
  • Author
  • Bit
  • 3 replies
  • January 26, 2022

Thanks for your reply

 

The source data is available, but the media library went offline because the repository (and the DDBs and Index Cache) went offline

 

Not new hardware. Old hardware that got disconnected since November 29, 2021. We found out that after the power issue, most of the zoning configuration was lost.

We had to rezone.

 

Now we have brought them back up. Unfortunately, the drive letters have changed. I tried changing them to what they were before, but the media library still remained offline.

 

The issue is NOT with the data on these partitions. It’s getting everything back up.

 

Let me share an example of the error I have been receiving:

 

Deduplication Database(DDB) access path [H:\DDB-3] on MediaAgent [*,*] is offline for Storage Policy Copy [Default File SP2 / Primary ]. Offline Reason: Thea ctive DDB of current storage policy copy is not available to use. Please wait till the DeDuplication DB Reconstruction job reconstructs the partitions.

I tried the reconstruction job, and it failed. I even attempted this without previous backup, same problem,

 

All these happened because the partitions went down after a power outage. It’s actually an emergency because the file server needs to be up as soon as possible.

 

Mike Struening
Vaulter
Forum|alt.badge.img+23

@zadok001 , the issue is definitely that the DDB can’t be seen, and when it is trying to perform a reconstruction, the drive is likely not seen as needed.

So right now, the drives are back online and the drive letters are corrected?

There’s a few things we could check, though this is definitely an emergency.  I would suggest creating a support case as soon as you can so an engineer can remote in.

Once you do, share the case number here so I can follow up accordingly.

I’d hate to work on the DDB being online only to find that the library isn’t accepting jobs either :nerd:

https://www.linkedin.com/in/michael-struening
L
1 person likes this
  • L
    Laurent
L
Byte
Forum|alt.badge.img+15
  • Laurent
  • Byte
  • 386 replies
  • Answer
  • January 27, 2022

1st, as suggested by Mike, call the support and they’ll be able to assist you. That’s an emergency and they’re handful for such.

 

DDB is only used to backup. 

From what I understood, you just need to restore data, not to backup.

So, of course, you’ll get tons of warning/critical errors from the Commserve/MA logs reporting a ddb or devices are down, but this should not prevent your disk library to be back online, if it had been brought up exactly as it was before the issue.

The devil lies in the details, and maybe here in your MA where you have you disklibrary, the LUN IDs could have changed, the windows disk IDs (SAN mostly equals to multipathing so after a hardware failure paths could have changed also), resulting in data inside the volume is OK, but from a topdown hardware volume point of view, the device hosting this volume(s) for Commvault is not exactly the same. Then Commvault reports that disk library as offline.

In this disk library, do you have more than 1 mount path ?

What is the status of the mount path : is it offline ? Enabled ? Compare this and details to the physical volume inside the OS.

And also make sure for this MA that all backup, pruning, data aging activities are disabled until situation is solved and you recovered your data.

 

Most helpful members this week

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings