Solved

Tapes wasted by the index jobs

  • 6 October 2023
  • 10 replies
  • 266 views

Badge +2

Hi community,

we have a lot of LTO9 tapes with only few Index Jobs on them (Jobs are in megabyte range) and I would like to re-use these LTO9 tapes (17.58 TB) for other tape-out copies. When these jobs will expire, nobody knows. 

 

Is there any way to copy still valid index jobs to another tape?
Pick for refresh, tape to tape copy…?

Thanks

icon

Best answer by Nutan Pawar G 6 October 2023, 12:32

View original

10 replies

Userlevel 3
Badge +8

Hi @clariion 

 

By downloading the Index Backup Retention Report, you can run the report against the Index Job ID and the results will tell you whether the Index Backup Job is protecting any backup jobs.

When protecting backup jobs, corresponding index jobs are required for browse and restore purposes.

https://cloud.commvault.com/webconsole/softwarestore/store.do#!/135/743/23630

 

If index jobs are not protecting any backup jobs or show orphan jobs, you can enable the parameters below and run a data aging job to see if it helps clean up index jobs.

https://documentation.commvault.com/2022e/expert/11019_media_management_configuration_data_aging.html

Delete archive files of indexing backup associated with deleted backup sets

Definition: Use this parameter to specify the deletion of the index backup information after the deletion of associated backup set.

Default Value: 0

Range: 0 (disabled) or 1 (enabled)

Delete checkpoints for not currently active storage policies

Definition: When this option is enabled, checkpoints (Index backups) from inactive storage policies will be pruned with data aging job.

Default Value: 0

Range: 0 (disabled) or 1 (enabled)

Usage: Set this option to 1 and run data aging job to prune index backups from inactive storage policies.

------------------------------------------------------------

For issues related to running out of tapes / multiple tapes held with only a single index backup on each, you can use the media refresh to merge the index backups onto one tape and reclaim other tapes back to the scratch pool

https://documentation.commvault.com/11.24/expert/10516_media_refresh.html 

Userlevel 3
Badge +6

hi @clariion 

we have exactly the same situation for months, and we have raised several support cases for this issue.
Currently the only workarround is to perform media-refresh operation on regular bases to consolidate those index to less tapes but it isn’t solving the actual issue..

We know based on the current support ticket that something is in the works to “age” index backups after a while as there is no valid backup’s anymore.

So suggest that you create a support ticket as well to get this issue priority and is more on the radar by product management and development.

Just performing media refresh actions and buying more tapes is not a solution i think..

 

 

Badge +2

Thank you guys @Nutan Pawar G and @M Scheepers.

 

At first I have created a report that was suggested by @Nutan Pawar G, but there was everything fine. There are indeed protected backup jobs in relation with index jobs. So I proceeded to perform media-refresh and it helped me to free up 40 tapes.

Thanks again.

Userlevel 3
Badge +6

I am sorry @Nutan Pawar G,

 

Your answer is the pink elephant view of Commvault ignoring multiple customers complaining about this very specific issue.

Due to the nature of indexing v2 (backupset level) the index backup is out-of-sync with the jobs we want protected for a longer period on an offline medium.

 

Now if you have a grandfather-father-son setup for tapes, which is best practice for years now, this means you will end up with lots and lots of tapes holding a index backup worth 10MB on a tape that could hold 18TB worth of compressed data is insane.

 

This has been a clear oversight when Commvault development wrote the layout for indexing v2 initially introduced in 11.14 and has been a painfull road mostly for customers still using offline (=tape) media.

I can state Commvault development have, reluctantly, been convinced to solve this better and there are 2 areas that will be changes:

  • For Indexing v2 (subclient level) you can now enable a flag under control panel to include a index database backup at the end of a full. This is very similar to how indexing v1 worked.
    NOTE1: This requires all supported subclient level indexing v2 clients to be converted using the workflow AND at this moment is only support on FS backups, VSA backups do not support this (yet).
  • In a future release Commvault will age off index backups on tapes as they are not meant to be used in that way in the first place (hence the oversight). This will trigger an index reconstruction if you do a browse and then ask for tapes to be returned to the tape library.
    NOTE2: There is absolutely no support in Command Center at this very moment to alert a user that tapes are needed, this is also expected to be improved in a future version.

 

In our case the tapes are with an external (third party) vendor, stored in their bunker and getting tapes back requires customer approvals, which is very hard to do every 1 to 2 months to clear up these tapes.

 

The above improvements and customer side actions should alleviate this issue in a future state.

Userlevel 1
Badge +1

Hello @mikevg 


I apologize for the delay here. We are currently in the process of arranging an engineering review for your clarification. Thanks for sharing additional details. Please rest assured that we are working diligently to provide you with a detailed response as soon as possible.

Thank you for your patience and understanding.

Userlevel 4
Badge +12

Praveen

Is there an expected release or timeframe when a V2 VSA Full backup will be able to ‘request’ a Full backup of the associated V2 Index

Userlevel 3
Badge +6

Hello @Praveen Joshua,

 

We are coming up to 2 months for an “engineering review”. Very curious to hear the findings, can you share that please?

Badge

Hello,

We also have the same problem with our offsite backup to TAPEs for a long time... After a lot of calls and tickets to CM support nobody provided a solution… They are unable or they dont want to understand the problem...

It is a very important issue and we are lost our trust to the product we are using for more that 10 years…

As a temporary solution we have created an Azure Cloud storage Library and we configured all the indexes to be saved on this Library. 

If there is any progress on this issue, please let us know.

The requirement is VERY SIMPLE. We just want to restore all our infrastructure in case of disaster from our "poor" TAPES.

Thank you

 

Userlevel 3
Badge +6

Hello @Georgios,

 

At this point I would suggest to point your Commvault account representative to this community post and request internal escalation (we did the same 2+ years ago, which is why I could share a bit above).

 

my 0.02,

 

Regards,

Mike

Userlevel 4
Badge +10

To add some context here;

This is a known challenge when using Index V2 caused by those index jobs protecting jobs in other tapes, they won’t therefore age unless the jobs they are protected are also aged.

We currently have CMR 404337 raised for this to enhance the logic, so index backup within a tape only protect jobs from such tape, allowing for the correct aging/pruning of the whole tape when no jobs within are valid anymore.

Reply