commvault.com commvault.com

Events
Platform Updates
Support
How-To Videos
Vault Club

Community
Commvault Cloud Topics
Storage and Deduplication

Storage and Deduplication

Discuss any topic related to storage or deduplication best practices

900 Topics
4,105 Replies

When you subscribe we will email you when there is a new topic in this category

900 Topics

Newest first

Recently active Most replies Most views

Hari OruppakkayilBit

asked in Storage and Deduplication

YYZByte

asked in Storage and Deduplication

Expire uncompleted "To Be Copied" - Selective Copy - First Full of the Year

I’d like to create a single beginning of year (Jan 2023) selective copy (to Tape)Settings are Selective : Yearly Full, First Full of the YearWhat I’d ideally like to achieve is to capture only those Full backups that took place in the first 2 weeks of the new year - which is ‘easy’ to do. What happens though is:If some ‘new data’ or new subclient is created and it obtains a first full in say February 2023, that data will ‘wait’ until it can be written to tapeBut no physical Tape will be made available until Jan 2024(eg 10 Tapes are put in on 1 January 2023 , 10 Tapes are removed 31 Jan 2023, and no further Tapes will be inserted until 1 Jan 2024) What I’d like to do is:If There are AUx Copies waiting (specific to Storage Policy and Storage Policy Copy), if the Aux has been waiting for more than (say) 60 days, is change the job to ‘DO NOT COPY That is ‘almost’ have an expiry data on waiting copies for Aux to Tape or an option that the ‘First Full of the Year’ has a validity period of

1 year ago

Donnie D

asked in Storage and Deduplication

AWS S3 - Dash copy between buckets and promote copy?

Hi, I could not find anywhere that adressed this, so asking here. I read that the only way to “migrate data” between storage classes would be as documented. However, can this be done? I have a Hyperscale as a Primary Copy I have an existing AWS S3 Standard bucket as a Dash Copy with Deduplication. I want to create another Dash Copy to an AWS S3-IA Bucket with Deduplication. I want to promote that copy to be the secondary copy and get rid of the existing bucket. Effectively, this seems like migrating the data just as good as going through the process described with the Cloud Tool. Am I wrong? Can this be done?

1 year ago

YolandaByte

asked in Storage and Deduplication

Verify DDB Reconstruction Jobs

Hello Community, I am new to Commvault. I am trying to check the status of a DDB Reconstruction failed job. I checked the Storage policy but I don’t see the job that created the internal ticket. Type: Job Management - DeDup DB ReconstructionDetected Criteria: Job StartedDetected Thanks.

1 year ago

barcodeBit

asked in Storage and Deduplication

Data Aging In AWS s3

Greetings, We have some Aux copies that go to our AWS s3 bucket. The storage policy this is under has a 30 day on prem and 365 day cloud policy. The 30 day on prem (primary) has data aging turned on and seems to be pruning and getting rid of jobs past 30 days. I took a look at the properties of the Aux copy job though and noticed that the check box for data aging was not selected. When I view all jobs for this Aux copy, it showed jobs back from years ago unfortunately. So that tells me that nothing is aging out or getting cleaned up. Our s3 bucket is getting very large and we need to clean up all of these old jobs to bring it down to a reasonable size. My question is how best to do this clean up? Can I view the jobs under the Aux copy and then just select all of them past our retention and delete? Would this delete data out of the s3 bucket also if I did this? I did select the data aging check box now and hit ok, then ran a data aging job from the commcell root and just ran it against

1 year ago

DaveKByte

asked in Storage and Deduplication

Auxcop Streams , what triggers them to increase during the job

I have an Auxcopy job running, it allocates 55 streams and as time progresses those streams gradually drop as streams complete. I now have my Auxcopy that has been sitting on 7 jobs for a couple of hours. When I kill the job and restart it I go back up to 61 streams. Am I improving things by doing this, will my job finish faster? Why does the streams not increase by itself during the job or why does it take so long to do so? Can I improve this?

1 year ago

barcodeBit

asked in Storage and Deduplication

WORM Retention with AWS S3

Greetings,We are trying to redo our Aux copy backups we currently have going to S3 now. We want to harden our Aux/cloud backups by creating new S3 buckets and turning on WORM for them. I created a new test bucket and turning on the S3 Object Lock for the bucket. My question is around retention times. We currently have and will continue our retention of our yearly (gathering all the weekly full backups for 365 days) backups and then have data aging start deleting the full backups on-hand after 365 days. If I want to keep this same retention with WORM turned on, do I need to set the S3 object lock retention to the same amount (or less/more than 365 days)? I am reading through documents and other posts about setting retention 2 or 3 times the amount as the policy to prevent accidental deletion or preventing it from being deleted. This is where I am confused on. Right now we do have it deduped going to our Aux copy/cloud, but I guess I question if we should even dedupe our new backups to t

1 year ago

Christian MoisanBit

asked in Storage and Deduplication

recopying contents from a tape that was physically dropped and is no longer usable

Hello! This morning I was all thumbs and dropped a tape. This caused the little pin inside to come loose and the tape was barely holding. I have to consider it dead now. Is there a way to flag the data inside to be recopied to another tape? Thanks!

1 year ago

Base64Byte

asked in Storage and Deduplication

Restore o365 content from tape

Hello, I have a question.Is it possible to restore Teams/Sharepoint or oneDrive content from tape? I can't browse the tape in Java GUI and I can't select the precedent in the command center. Is there any way to do a restore from tape?

1 year ago

KevinDodd

asked in Storage and Deduplication

Data verification job with S3 storage

Evening folksI am in the process of enabling data verification across our storage policies - some of our servers are EC2 instances with Commvault configured to backup directly to S3.I assume if I were to enable data verification in the above scenario I would incur further charges as Commvault would be reading data from the S3 bucket?Also, I just want to check my workings out is correct, or there about. If we are charged for using data verification in the above scenario - if, for example, the data verification job needed to verify 150GB worth of data my math would be 150GB (size of data) / 32MB (Block size that CV stores data in) / 1000 (S3 charges per 1000 requests) * price per 1000 READ requests? Many thanks

1 year ago

RobsterFineBit

asked in Storage and Deduplication

Planning / sizing for cloud storage tiers and documentation question

Hi,our documentation around combined storage tiers in cloud could be a bit misleading around sizing for Warm and archive tier storage requirements.See in red below.what’s the ideal ratio to size? (or do we need to just go 10% on warm tier to be safe?)https://documentation.commvault.com/2022e/expert/139246_combined_storage_tier.html BackupsCommvaults combined storage tiers works by placing Commvault metadata used in both deduplication and non-deduplicated backups in the warmer or frequent access tiers. This allows you to perform a simple Browse and Restore of the archival data, without the delay of a cloud archive storage recall.As a result, more than 90% of the backup data gets stored in the Archive tier, while only up to 10% of the data is stored in the warmer tiers. Deduplicated and non-deduplicated data is supported in Commvault combined storage tiers. For deduplicated data, the backup indexes and deduplication metadata is stored in the warmer tier. For non-deduplicated data, th

1 year ago

drPhilByte

asked in Storage and Deduplication

Multiple deduplication engines within one disk drive

Hi all,I have a following question. If I needed to have more disk storage pools (in order to have more storage policy copies with different retention), I would need to have more deduplication engines. Is it possible to create multiple deduplication engines within one disk drive? Are there any limitation for the number of deduplication engines on one disk drive? Thanks in advance for your engagement!

1 year ago

RobsterFineBit

asked in Storage and Deduplication

How to read Azure ZRS blob is primary zone has failed.

I’m looking for some guidance or documentation on what can be done with ZRS storage. Say we have the following availability zones in the region.Zone 1: Media Agents and Hot blob backing up local VM’s, files etc. no Aux copy.Zone 2: Standby Media Agent and conserve powered down. Commsere DR recovery done manually. Zone1 failed so we power up MA and CS at Zone2 - perform manual DR backup set restore Next step - getting the ZRS storage mounted as primary. - any links, docs steps would be much appreciated.Thanks,Regards,Robert

1 year ago

Dave from RenoByte

asked in Storage and Deduplication

Seeding a Deduplicated Storage Policy

Has anyone ever transferred their initial baseline backup between two sites using an available removable disk drive (Seeding a Deduplicated Storage Policy). We are looking like may have to use this method to get a good AUX copy for some remote sites. How did it go? did you have any issues? Thanks for your input.

1 year ago

JeremyByte

asked in Storage and Deduplication

Error occurred in Disk Media Path (Permission Denied)

Hi,We keep getting the following alerts in the “Anomaly Report”:It seems clear. The Media Agent is unable to access the Mount Path over SMB due to a permission issue. Backups don’t seem to be impacted so it feels like Commvault is doing a retry, after which is works fine. But we would like to know where this error is coming from. It doesn’t happen frequently.We checked the Event Viewer on the Media Agent, and we do see an SMB error, but the time doesn’t match, the error on Windows is 30min before the error in Commvault. We checked the Storage where this SMB share comes from but also nothing special to see here.I wanted to check with the Community to see where this might come from and how to troubleshoot this properly. Would this an issue with the storage, or rather the Media Agent on Windows? Or maybe an issue in Commvault?

1 year ago

RobsterFineBit

asked in Storage and Deduplication

HyperScale Migration using Auxiliary copy to Azure / Metallic storage - DDB block size

Hi,I’m working on a project where a large Hyperscale environment needs to be migrated to Azure. Looking at using either Azure storage or Metallic recovery reserve and/or possibly a mix of the two. There’s short term 30 days as well as LTR 5 to 7 years so will use Hot / Cool tiers (possibly combined storage tiers if Metallic not used)HS is setup using the default DDB block size 128KB. In an ideal world - one could just setup a secondary copy for the cloud storage (either Metallic or Azure) and kick off the Aux copy to cloud, just let it run to get the data over in to Azure then eventually promote it to primary copy… however…As the cloud storage will then be used as a primary copy - ideally, we want to configure it with 512KB DDB block size. Media agents will be setup in Azure as they will eventually become the production MA’s once things get cut over. Some key questions on the above:copying between storage policies with different DDB block sizes – how will this affect overall dedupl

1 year ago

Mohit ChordiaByte

asked in Storage and Deduplication

Disk Library Mount path space usage monitoring

HI Team,Is there any API or report using which we can monitor individual library mount paths ?CV alerts are for complete library but i need to configure an alert specific to mount paths( coming from different media agents or servers ) part of same disk library. We receive below alerts when one or few of the mount paths of library met reserved space .Failure Reason: Insufficient disk space. Available mount paths are not enabled for write or have met reserved space limit. Enable/add more mount paths or add more disk space to existing mount paths. Please check mount paths on the library Need to configure alerts on mount path level so that we can disable the mount paths in advance .Regards, Mohit

1 year ago

jxbarbBit

asked in Storage and Deduplication

MCSS Library performances

Hello, I have a matter that could require some help.We have a primary copy on a disk Library, a secondary on another disk Library and a third copy on a MCSS cloud Library. All copies have dedup enabled.Auxiliary copies run fine between the disk librairies but when it comes to sending the data to the MCSS library it takes forever. We tried to increase the number of streams, use either of the disk libraries as source for the aux copies but we can’t achieve suitable performances. What we see is that the process generate excessive read operations on the source library. Dedup block size is 128 on the disk libraries and 512 on MCSS.Commvault version 11.24.Any help would be appreciated.Regards, Jean-xavier

1 year ago

pma

asked in Storage and Deduplication

Linux Disk Library Volume Sizing Best Practices (Design)

So,I’m having a project with Commvault and a HPE Apollo 4200 with 24x 12 TB NLSAS drives.What would be a best practice approach when creating the array(and logical volume) in the aerver Raid Controller?I read some people saying that the max volume should be 4 TB per mount point, does this make sense these days ?If yes, that means that I need to create 66 volumes of 4TB each within raid controller.Any beneficts on doing this versus a single volume with 264 TB?thanks

1 year ago

MauroByte

asked in Storage and Deduplication

Device_Label deleted

I understand the DEVICE_LABEL file is used for pruning so it missing could be an issue.I have a customer who's deleted the file off a volume.Will this affect pruning and would there be any other potential issues that could arise?Is there a way to recreate the file if it is required, without recreating the volume?The volume is configured as a WORM device for compliance.Thanks so much.

1 year ago

Bjorn MByte

asked in Storage and Deduplication

MediaAgent Deletion Impact Report shows an association that is not there ?

Hi!We’ve recently moved media-agents to new servers, and i’m now in the process of “cleaning up”. And for 3 of the 4 media-agents the report showed nothing, and i could delete the mediaagent without any issue. However, for the last media-agent, it complains about “Proxy Server to Perform Intellisnap Backup Operations”, and lists a client name, a backupset name, and a subclient name.This was “correct”, because there was a subclient there with that name that was configured for intellisnap (but not in use, and had no backups). So i deleted the subclient… but even so, now several days later, when i try to delete the media-agent, the report still complains about the same “issue”. I could ofcourse just ignore the “warning” and proceed with the deletion, but i’d much prefer for the report to show that there is no association, before proceeding with the deletion.

1 year ago

Bjorn MByte

asked in Storage and Deduplication

Unusual performance drop detected in pruning for following deduplication databases

Unusual performance drop detected in pruning for following deduplication databases: [GDSP_ALL_<Servername_8]: due to increase in (CommServe Job Records to be Deleted) - Check details in: http://backupserver:80/webconsole/reportsplus/reportViewer.jsp?idstringIf i check the report, it’s grown from 0 to 12399 commserve job records to be deleted from 9’th january to today the 18’th.….However, this media-agent is a media-agent that is spending half it’s life completely offline and shut down, and it has been shut down since the 9’th, and powered on again today, so i assume this is expected behavior, but because i have not seen this message before, i figured i’d check with the hivemind first :-) Software version is: 11.24.78

1 year ago

tigger2Byte

asked in Storage and Deduplication

How to properly deleting/decommissioning mount points associated to old storage: DDB's appear still associated with the mount paths.

We have added new storage to Commvault, and set the old mount paths to “Disabled for Write” via the mount path “Allocation Policy” → “Disable mount path for new data” + “prevent data block references for new backups”All mount paths that are “disabled for write” do not have any data on them via the “mount path” → “view contents” option.We have waited a several months for all the data to age off.BUT…I see info on the forums/docs that “data’ may still be on the storage, and there are referenced to “baseline data” in use by our DDB’s. When I go to the mount path properties → Deduplication DB’s tab, I see that all our “disabled for write” mount paths have DDB’s listed in them. So it appears commvault is still using the storage in some way. I saw a post that indicated “The LOGICAL jobs no longer exist on that mount path, but the original blocks are still there, and are being referenced by newer jobs. For that reason, jobs listed in other mount paths are referencing blocks in this mount pat

9 months ago

Page 12 / 36

Badge winners

Keith Kirbyhas earned the badge Topic Starter 1
arikhas earned the badge Topic Starter 1
Suzanny Lloydhas earned the badge Contributor 10
Heemansu Gehlothas earned the badge Contributor 1
Abhishek Narulkarhas earned the badge Solver 1

Show all badges

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

Commvault Login

Login to the community

Commvault Login

Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.

Username or e-mail

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.