commvault.com commvault.com
Solved

DDB and Index Cache restore failing

  • 25 January 2022
  • 4 replies
  • 931 views
Z
Rank
Badge +2

Hi

 

Good day

 

So there was a power outage, and I lost the SAN LUNs I provisioned for both DDB, Index Cache and even the storage repository. They were mapped/zoned to the Media Agent (Server) initially.

 

Unfortunately, the guy on ground provisioned new storage for these DDBs AND Index Cache, changed the drive letters and reassigned the partitions

 

I have brought back the LUNs and the backup files within are still intact.

 

However, the location specified in the media library is offline.

 

I can’t restore anything from this location.

Please I need help, because we need to restore a file server

icon

Best answer by Laurent 27 January 2022, 10:57

View original
If you have a question or comment, please create a topic

4 replies

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

To confirm, when you say you can’t restore anything from this location, are you saying the restore is failing now (because it is not connected to the drives properly), or you can’t restore the actual source data at all?

The crux of the issue is what data you can pull of the downed drives.  If the data is there, can you return the LUNs back to their proper drives?

It sounds like (and confirm) that you have new hardware connected, but with new drive letters (and they contain no data).

You should be able to restore the data and fix the drive letters and everything should (in theory) work.

However, before we can determine the best way forward, we need to know what things look like now, and what you can and can’t get the data from.

https://www.linkedin.com/in/michael-struening
Z
Rank
Badge +2

Thanks for your reply

 

The source data is available, but the media library went offline because the repository (and the DDBs and Index Cache) went offline

 

Not new hardware. Old hardware that got disconnected since November 29, 2021. We found out that after the power issue, most of the zoning configuration was lost.

We had to rezone.

 

Now we have brought them back up. Unfortunately, the drive letters have changed. I tried changing them to what they were before, but the media library still remained offline.

 

The issue is NOT with the data on these partitions. It’s getting everything back up.

 

Let me share an example of the error I have been receiving:

 

Deduplication Database(DDB) access path [H:\DDB-3] on MediaAgent [*,*] is offline for Storage Policy Copy [Default File SP2 / Primary ]. Offline Reason: Thea ctive DDB of current storage policy copy is not available to use. Please wait till the DeDuplication DB Reconstruction job reconstructs the partitions.

I tried the reconstruction job, and it failed. I even attempted this without previous backup, same problem,

 

All these happened because the partitions went down after a power outage. It’s actually an emergency because the file server needs to be up as soon as possible.

 

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

@zadok001 , the issue is definitely that the DDB can’t be seen, and when it is trying to perform a reconstruction, the drive is likely not seen as needed.

So right now, the drives are back online and the drive letters are corrected?

There’s a few things we could check, though this is definitely an emergency.  I would suggest creating a support case as soon as you can so an engineer can remote in.

Once you do, share the case number here so I can follow up accordingly.

I’d hate to work on the DDB being online only to find that the library isn’t accepting jobs either :nerd:

https://www.linkedin.com/in/michael-struening
L
Rank
Userlevel 6
Badge +15

1st, as suggested by Mike, call the support and they’ll be able to assist you. That’s an emergency and they’re handful for such.

 

DDB is only used to backup. 

From what I understood, you just need to restore data, not to backup.

So, of course, you’ll get tons of warning/critical errors from the Commserve/MA logs reporting a ddb or devices are down, but this should not prevent your disk library to be back online, if it had been brought up exactly as it was before the issue.

The devil lies in the details, and maybe here in your MA where you have you disklibrary, the LUN IDs could have changed, the windows disk IDs (SAN mostly equals to multipathing so after a hardware failure paths could have changed also), resulting in data inside the volume is OK, but from a topdown hardware volume point of view, the device hosting this volume(s) for Commvault is not exactly the same. Then Commvault reports that disk library as offline.

In this disk library, do you have more than 1 mount path ?

What is the status of the mount path : is it offline ? Enabled ? Compare this and details to the physical volume inside the OS.

And also make sure for this MA that all backup, pruning, data aging activities are disabled until situation is solved and you recovered your data.

 

Terms & ConditionsCookie settings