Solved

Index version 2 backups

  • 26 January 2021
  • 7 replies
  • 4460 views

Userlevel 2
Badge +8

I am a bit unclear with Index version 2 since version 11.  If we have a new Mediaagent with the latest Commvault agent installed; as I understand it will utilize Index version 2; what happens with the clients which do not support this version.  

Also, we have a selective policy wherein the server last monthly full is taped out; what about the Index version 2 database backups; As I understand only action logs are backed up as a part of the backup process; what happens to the Index database; so if we were to restore an historical six month old backup with Index version 1; everything will be restored from the backups as it is backed up as a part of the backup job; however what happens if it is Index version 2

I do notice that Index database is backed up as a part of “Big-Apps” against the storage policy; however; I cant see the historical jobs going to tape  

icon

Best answer by Damian Andre 26 January 2021, 22:06

View original

7 replies

Userlevel 7
Badge +23

Hi @Theseeker 

Media agents are indexing version agnostic - they can all handle both V1 or V2 indexes at the same time, so no issues there. Typically new clients will default to using V2, whereas old clients may need to go through an upgrade procedure to get them to V2, or a new pseudoclient has to be created in some cases (i.e for VMware/virtualization clients). Also, check out the available reports as they can be really helpful.

With V2, each backup job (full, diff or incremental) will generate ‘action logs’ - this is the index for the current job, and the current job only. These action logs are protected in each job, similar to V1. If you lost your index, and did not have access to an index backup, Commvault can use all the action logs to reconstruct the index database from all the jobs - however, this is time-consuming and for large backupsets it can take a while before you can begin your restore. In contrast, V1 indexing was a FULL index database backup included with each and every job. That used considerably more space and slowed down operations because we’d have to maintain a larger database synchronously, but the benefit was quick recovery should you lose your index.

Fortunately, backing up action logs with V2 is only a failsafe - the index backup happens at the “big apps” level as you noted. This is an independent backup and will additionally backup the full index database for each subclient/backupset (depending on the agent). If the index is lost, this backup allows for quick and seamless recovery without having to reconstruct much at all (perhaps only the last few jobs after the backup was done).

So if you need to restore 6 months ago, that should be in the active V2 database on the media agent without issue. If you lost your index cache and wanted to restore from 6 months ago, then the index backup is automatically restored, and any outstanding transaction logs are applied to make it operational for browse & restore - but this is a very unlikely scenario unless you have a DR situation.

If you are using tape then you certainly want to ensure your indexing backups are successful. If the index backups are not available then you can still restore, but it's going to need to pull at least a full cycle worth of tapes to reconstruct the index, which is not ideal. There is logic to ensure a copy of the indexing backups are available everywhere your backups are for this very scenario.

I am not sure if we will write V2 index backups to selective copies - I think that was planned but not sure about the current status. Will need to check internally ...

 

 

Userlevel 5
Badge +13

In v11 SP11, we introduced the Index backup for the subclients associated to a particular Storage policy to be copied from primary to secondary copies automatically. So for each storage policy we ended creating an index server for protecting the index v2:

https://documentation.commvault.com/commvault/v11_sp20/article?p=10784.htm

“The index backup process, which occurs at the backupset level, was changed in SP11 to better protect a CommCell during disaster recovery events. The system now auto-creates one IndexBackup pseudoclient for each storage policy associated with an Indexing Version 2 client (created in Client Computer Groups > Index Servers in the CommCell console), and backs up the full index per storage policy. This change ensures that the system stores an index backup with its associated data backup on secondary storage, thus providing optimal recoverability of data in disaster recovery situations.”

 

I do hope that @Theseeker CommCell is in an “updated” version, in some customer I have seen that they edit the secondary properties and filter some subclients like the Big-Apps that takes care of the index backups, not allowing its jump to secondary level, this is not good.

 

Other reason can be your clients are using v1 index for a particular storage policy, no matter if your Media Agent is brand new install. so no index backup is needed.

Userlevel 2
Badge +10

@Damian Andre

Question: Above in the answer you indicate “If you are using tape then you certainly want to ensure your indexing backups are successful.”  Does this imply that, if you are using tape, you probably do want to write these index backups to tape?  I have a system where this is occurring (index backups are being written to tape) for all the indexserver backups and I wasn’t sure if this is the default behaviour to be written to tape or if another admin did it long ago.  It sounds like writing them to tape is not a requirement just if you want to restore an old tape it is a benefit to also have the index on tape to make the restore index process faster as well as to not have to hunt down many tapes if they are on offsite/long term storage.

Userlevel 7
Badge +23

I’ll defer to @Damian Andre on his intent, though I suspect he is just saying that if you don’t have the index available, then we will have to recall tapes from the various backups to reconstruct the index which can be cumbersome.

Better safe than sorry, basically :nerd:

Userlevel 7
Badge +23

@Damian Andre

Question: Above in the answer you indicate “If you are using tape then you certainly want to ensure your indexing backups are successful.”  Does this imply that, if you are using tape, you probably do want to write these index backups to tape?  I have a system where this is occurring (index backups are being written to tape) for all the indexserver backups and I wasn’t sure if this is the default behaviour to be written to tape or if another admin did it long ago.  It sounds like writing them to tape is not a requirement just if you want to restore an old tape it is a benefit to also have the index on tape to make the restore index process faster as well as to not have to hunt down many tapes if they are on offsite/long term storage.

 

The system should try write a copy of the index backup to every copy in the storage policy by default, but it wasn't always this way. - the reason is that you always want the index nearby for every copy - otherwise it has to pull the index transactions (called action logs) out of each backup to reconstruct the index database - that’s not something you want to do with tape as it would be lengthy - much better solution is to pull back the backup and only recover the incremental action logs. 

Badge +2

Hello,

This is a test I just tried to do: restoring all data only from tapes.

Usually, full backups are copied to tapes every week, so I took a tape set from 2 weeks ago as it was exported.

I restored the CommServe from Commvault Cloud.

New Media Agent and tape library were deployed and the tape set was inserted in the library.

Index v1 client recovery works fine.

Index v2 client recovery asks for tapes that are out of the current set to recall indexes, so I can’t restore anything.

I tried the catalog option but it takes days so it’s not viable.

 

Is there a way to ensure that a weekly tape set contains also the necessary index backups?

 

Badge +2

Is it possible to increase the number of streams for indexing ?

 

My indexing is really slow and only using 5 streams …

 

regards

Reply