commvault.com commvault.com

Storage and Deduplication

Discuss any topic related to storage or deduplication best practices

  • 892 Topics
  • 4,088 Replies

892 Topics

V
Bit
VedamurthyBit
asked in Storage and Deduplication

Could not connect to the DeDuplication Database process for Store Id [1].

Backups failing with error,  Could not connect to the DeDuplication Database process for Store Id [1]. +   unable find any KB article for the same,      can some1 suggest on this

1
MichaelCapon
Vaulter
3 years ago
R
Bit
RolandasBit
asked in Storage and Deduplication

Object storage vs File storage (S3 vs NFS)

Hi all, we are looking into our backup strategy and investigating few scenarios of backend storage. Right now we are considering object storage via S3 protocol and file storage (JBOD) over NFS protocol. The data which will be send there is on-prem filesystems, databases, VM’s etc. Total capacity - over 10 PB.Currently we tested some object storage over S3 protocol but we faced issues with data reclamation (garbage collection for expired objects taking way too long and capacity reclaimed wait time takes over month or few). Can you share your experience with back-end storage, what challenges you faced or how you solved my mentioned issues, also, what advantages you see comparing S3 & NFS protocols for backups. All feedback is very appreciated.Thanks! 

4
Mike Struening
Vaulter
2 years ago
J
Bit
JasonTFBit
asked in Storage and Deduplication

Data Domain DDBoost feature

My company is planning to do a tech refresh on our aging data domain to a newer version. We also highlighted that we’re having backup slowness issues on some of our large Oracle databases & some NDMP backups. Our current configuration is backing up to a VTL located in our data domain & no compression or deduped are enabled from Commvault layer.Dell’s sales team advice us to purchase an additional license for DDBoost for the new Data Domain. This is because DDBoost will be able to perform a very good dedupe & compression rate from the source level before transferring it to our data domain, thus saving the time it took to transfer via network.However I’ve been checking around Commvault’s KB and looks like commvault only works with BoostFS & not DD Boost. I haven’t check with Dell yet regarding this. May I know has anyone implemented DDBoost on your environment for backing up databases/vm & NDMP? 

24
Onno van den Berg
Commvault Certified Expert
1 year ago
Ken_H
Byte
Ken_HByte
asked in Storage and Deduplication

How to show if my backups are encrypted

The auditors want to see if my backups are encrypted and I’m not sure where to go in the CommVault GUI to show that.  I don’t see anything about encryption in the properties for my storage libraries or my storage policies.  Where do I show whether or not my backups are encrypted?Ken

17
Onno van den Berg
Commvault Certified Expert
8 months ago
D
Byte
drPhilByte
asked in Storage and Deduplication

Description: Error occurred while processing chunk - error Code: [13:138]

Hi all!could you advice me how to troubleshoot following types of error: Error Code: [13:138] Description: Error occurred while processing chunk [xxx] in media [xxx], at the time of error in library [disklib01] and mount path [[xxx] /srv/commvault/disklib01/xxx], for storage policy [XXX] copy [Xxx] MediaAgent [svma1]: Backup Job [xxx]. Unable to setup the copy pipeline. Please check connectivity between Source MA [svma1] and Destination MA [svma1].  At a glance, it seems that it is not possible for CV to process chunk from the (index?)/disk library...However, the issue is connected with storage policy copy, that moves data from the disk library to the tape library (secondary copy). The main problem for us is that it is not possible to copy data to the tapes. Therefore, it may say Unable to setup the copy pipeline. The media agent is one server/device, that communicates with both disk and tape library. Lastly, the files in the related directories dont seem to be corrupted...Any suggesti

21
A
1 month ago
D
Byte
drPhilByte
asked in Storage and Deduplication

Cloud storage adding and troubleshooting

Hi there,I am trying to add a new cloud storage (S3 compatible), however I am unable to do so. Moreover, I don’t see any asociated log files. I only see this error message:“Failed to verify the device from MediaAgent [xxxxxx] with the error [Failed to check cloud server status, error = [[Cloud] The server failed to do the verification. Error = 44037]]. “ My question is: Which logs should I add to Logging setting? And how to troubleshoot this stuff in general? PS. As workaround I used this - https://documentation.commvault.com/commvault/v11/article?p=51230.htm but it didn’t help. Thanks for any ideas

5
Mike Struening
Vaulter
1 year ago
I
Bit
IgorBit
asked in Storage and Deduplication

After data clean-up and aging the Disk Library shows the same Free Space??

Hello all,I ran into a bit of an issue… Yesterday, one of the Disk Libraries got filled up and the backups went into waiting status. After haveing a look at the utilization, indeed it turned out to be 99,7% full.The main culprit were SQL server backups: There were some backup jobs with an extended retention - so I’ve deleted those, and some more of the old backup jobs to make space. I also ran Data Aging and could clearly see data chunks being deleted in SIDBPhysicalDeletes.log and after a while I got this:  So, I assume quite a bit of data was deleted. The Primary copy (blue) went from 52.95TB down to 19.81 TB. However, when I check the Free Space on the Library I got very little: So I checked the Mountpaths Space Usage for that DL: Data Written corresponds to amount of space used by the Primary copy: 19.8 TBHowever Size on Disk, which takes into account Data Written + aged jobs which are still referenced by valid jobs is still very high. Almost unchanged. I am quite confused by this.

11
Mike Struening
Vaulter
3 years ago
S
Vaulter
SamwiseVaulter
asked in Storage and Deduplication

I want to reclaim disk space using the DDB space reclamation feature, but how can I see how much space can be reclaimed?

I am running low space.  I want to run disk reclamation. If I run this, how much space can I be saving?

1
Emils
Vaulter
3 years ago
D
Byte
drPhilByte
asked in Storage and Deduplication

Drives in the tape library went offline

Hi there, could you please help me what to do if all drives within the tape library went offline? The offline reason seems to be very strange: [Cannot communicate with Media Mount Manager Service.Please ensure that.a. The MediaAgent is reachable from CommServe.b. All MediaAgent services are running.] A - Checked that MA is reachableB - Checked that all services are running What can help us to make drives online? Also verified in Windows device manager that drives are present and visible.

3
Mike Struening
Vaulter
2 years ago
Nikos.Kyrm
Byte
Nikos.KyrmByte
posted in Storage and Deduplication

Auxiliary copy VS Dash copy

Hello to all!In case of jobs replication (secondary copy), I see that these 2 options are available: Auxiliary copy & Dash copy.Whats the pros and cons of each?Dash copy is for deduplicated data (with Commvault dedup engine) and the Auxiliary copy for the rest “transitional data” / jobs?Lets share your thoughts.Nikos

3
Jos Meijer
Commvault Certified Expert
1 year ago
S
Vaulter
shaleyVaulter
asked in Storage and Deduplication

How do I troubleshoot slow backups or restores coming from or writing to disk?

I have general slow performance from my backend storage.  How can I determine the issue on my backend storage?

6
Mike Struening
Vaulter
2 years ago
L
Commvault Certified Expert
LukaszCommvault Certified Expert
asked in Storage and Deduplication

Delete Mount Path associated to DDB

Hello there,I’m have a minor issue, I cannot delete unused Mount Path, since it’s used by DDB. There’s a few of MPs under Disk Library dedicated to this DDB. In the DDB properites I can only remove whole Disk Lib, which is not the point.  CommCell says that in order to delete this MP, I need to delete each Storage Policy Copy which is referencing to this Disk Lib. It’s not an option neither.   Logs are saying something similar: EvMMConfigMgr::onMsgConfigStorageLibrary() - Error [470, Mount path is used by a Deduplication database.] occurred while deleting the mountPath[xx] ###### MLMMountPath::deleteMountPath() - :mlmmountpath.cpp:6170: Failed to delete mountpath [xx] due to error [470, Mount path is used by a Deduplication database.].###### MLMMountPath::deleteMountPath() - :mlmmountpath.cpp:5593: Failed to delete MountPath from database for Id [xx] due to error Mount path is used by a Deduplication database.:470  Do you have ideas, workaround to delete single MP in that situation? 

14
Mike Struening
Vaulter
2 years ago
M
Bit
Michal DolezalBit
asked in Storage and Deduplication

Could not connect to the DeDuplication Database process for Store Id

Hi, I am facing error regarding connection to DeDuplication Database. It can be caused by many reason but my specific problem is that the directory with DDB is locked for write. This was confirmed by SIDBEngine.log and simply you cannot create folder in DDB folder.Status of DDB is ONLINE. I can freely read from it, I can freely travers through it, I can copy the folder to different place and here I can create files/folders inside without limit. There is no way to manage from CMV - it is not possible to backup, restore, seal, verify etc. I cannot move the partition. It seems as Windows/Hardware problem but….I checked windows logs - no events from disk, scsi etc.I checked filesystem - all looks fineI checked hardware events (it is HPE server, so iLO log, IML log, I checked smart array status)It seems that it is limited only to this DDB folder. On the same drive, same volume, different directory all working as usual. So there is no write protection on drive or volume level (checked by dis

6
L
Byte
2 years ago
F
Byte
Farmer92Byte
posted in Storage and Deduplication

What type of storage array do you use?

We currently use IBM V5000 arrays for our Commvault backup target to land our deduped backups.  We are starting to review other options to see what other fast, cost effective options are out there.  I do prefer to use Fiber Channel connections, but open to options.   Since Commvault is really the brain in our scenario, the storage array does not really need any features, just good speed. What Vendor Storage arrays do you use?  Are you happy with it?  

31
S
Byte
7 months ago
E
Byte
Egor SkepkoByte
asked in Storage and Deduplication

DDB Full

One of our DDB getting full, what is the best practis to make some space on DDB? 

13
Mike Struening
Vaulter
2 years ago
A
Byte
Allan0105Byte
asked in Storage and Deduplication

Mount path is showing offline

Hi Team,we have a MA, which has four mount points all comes from backend SAN. out of four, one mount points are showing error while writing the data. i checked the disk management, disk was showing as read only than i have rebooted the MA and made the disk read/write. created test folder on the disk which working fine. but when i run the storage validation, it gives below error. did anyone had these kind of issue ? 7944  173c  09/09 10:20:44 685427 Scheduler  Set pending cause [Failed to mount the disk media in library [LIB05] with mount path [C:\CommVaulttLibrary\503] on MediaAgent [cv5]. Operating System could not find the path specified. Please ensure that the path is correct and accessible.

9
H
Bit
1 year ago
DaysAndCycles
Byte
DaysAndCycles
asked in Storage and Deduplication

Secondary Copy on Disk - Implications of WORM setting

Commvault 11.18 (soon to be 11.20).    - We are on the cusp of eliminating our secondary backups to tape.   The benefit of a secondary copy on tape was the build-in air-gapping (and ability to move it offsite for safe-keeping).    We plan to move to creating our secondary copies on disk in a different city.   Commvault’s built-in Ransomware protection is  no-brainer, but WHAT ABOUT WORM?    What are the implications of the WORM storage on space consumption?   Is there any scenario in which a WORM-enabled deduplicated secondary copy that is a true copy of the deduplicated primary copy (and with the same retention) would be any LARGER than the primary copy?   Presumably, if a 1000 jobs share one block on the secondary storage, that block will not be removed until the last of those 1000 jobs ages out.Any info is appreciated.   Thanks!

12
Nikos.Kyrm
Byte
1 year ago
B
Byte
bc1410Byte
asked in Storage and Deduplication

Critical Event - DDB reaching QI time threshold...

Still a noice commvault user..  Noticed that we have been getting the following Event in the evvent viewer.. DDB(s) reaching QI time threshold for adding new Subclients [{DDB [XXXXXX-GlobalDedup_Databases_5] MediaAgent [XXXX-cv1a] Path [D:\XXXX-DDP] AvgQITime [1153] Free Space Percentage [94] }]. New DDB(s) may gets created if it crosses threshold. Please review the DDB configuration, for more details refer Deduplication Building Block Guide on Books Online. Can anybody help me understand this and how to correct this.   Our small environment is all on AWS. Thanks BC 

5
M
Vaulter
2 years ago
D
Byte
drPhilByte
asked in Storage and Deduplication

DDB database reconstruction and deletion

Hi there, I want to share my experience with DDB reconstruction. My colleagues started the DDB database reconstruction because the DDB database was not in good condition. There were no current backups of DDB database (the last backup was 14 days old) and maybe some other error messages were active. They decided to start full reconstruction, maybe there was no option to make manual backup of the DDB database - hm, but the database had not been updated with new records anyway. The thing is that the reconstruction is very time consuming and moreover, since the DDB database is not running the backup jobs are not possible. The workaround we did was to temporarily disable deduplication. Another caveat is that we are running out of disk space. And that is how a disaster looks like . At the end I wil put my hypothetic questions. Is the DDB database needed for restoring of data - I wouldnt say so. And what would happen if the broken DDB database was deleted and completely the new one was built

1
Mike Struening
Vaulter
2 years ago
F
Francisco_Vasconcelos
asked in Storage and Deduplication

DDB corrupted on Windows server and sealed DDBs

Hello,  I would like to request your help as i’m quite new and have only basic knowledge of backup systems.After reading the commvault documentation available online and tried to troubleshoot the issues, they were narrowed down to 3: Sealed DDB are not aging out. When I try to run a verification of existing jobs on disk and deduplication database it says DDB is corrupted. we have our disks full, as the backups all have stopped.When i run the data retention forecast and compliance report it says “BASIC CYCLE” as reason why jobs are not aged out, i have this dedup policy to age out jobs after 1 cycle, so i guess that if i do a full backup, the previous will be aged out.. except i dont have any available disk space. Also I was unable to find a DDB backup as it seems there was never one to begin with. Should i reconstruct one new from the group up? How can i reduce the size of sealed DDBs as they are quite old.

12
Anthony.Hodges
Commvault Certified Expert
3 years ago
V
Byte
virgilByte
asked in Storage and Deduplication

error 13:138 on auxiliary copy between 2 MA

HelloI have a problem of auxiliary copy between two media agents with HP StoreOnceI have a 2*100MB WAN link between the two MA.I got 13:138 errors but job is still runningSee attachments for detailsIf someone could help meSP11.22No firewallingFirewall is Off on the two MA Regards  

1
J
Vaulter
2 years ago
D
Byte
djustisByte
asked in Storage and Deduplication

The SDT data transfer was terminated on a request from the Job Manager on 11.20.40

Running v11SP20.40.  We’re having a ton of occurrences in the last 3 weeks of this error across multiple environments:[The SDT data transfer was terminated on a request from the Job Manager.]I work in a MSP environment with the CS in one datacenter and the MAs spread out around the country.  We have a minimum of 10GB on our datacenter links from MA-MA and MA-CS.  I’m curious if anyone has seen this and has any resolution or troubleshooting steps.Thanks!

5
Damian Andre
Vaulter
3 years ago
S
Bit
Steven503Bit
asked in Storage and Deduplication

Migrating to new Media Agent Server

I’m looking to migrate to new Server Hardware for one of my Media Agents and looking for best approach and minimal downtime.Was thinking I could setup the new hardware with the media agent role/software and start to move mount paths to the new server.  Once all mount paths have been moved to the new MA I can then update my storage policies to point to the new MA.  Is there anything else I need to look out for or would need to do?Any advice or knowledge on this would be great.Thanks 

3
R
Byte
5 months ago
M
Byte
MountainGoatByte
asked in Storage and Deduplication

Sealing a DDB - what happens to the old DDB?

Hi Commvault-people. I have a large partitioned DDB which has been writing to a Cloud-based library, and has been for some time. The DDB partitions are roughly 2 TB’s in size. As is recommended when you have been writing to Cloud libraries, it should at some point be sealed, and I would like to go ahead. We are also on the cusp of the maximum threshold for Q&I times. However, I need to make sure I have enough space for my DDB on the current volumes. So the question is, what happens to the old DDB? I am assuming that it will remain at 2 TB’s in size until there is a corresponding reference for the blocks in the new DDB, or the blocks eventually age and are therefore not required. Maybe that will take months. Quite probably. Whilst it ages out old blocks, will the old DDB reduce in size? But what can I expect from the new DDB? If I only have a 3 TB volume, and 2TB is taken up by the old DDB then I really only have 1 TB available. if anyone has recently been through this scenario, it

10
Mike Struening
Vaulter
1 year ago
DaysAndCycles
Byte
DaysAndCycles
posted in Storage and Deduplication

Commvault Official Deduplication Limits

I think there are several deduplication-related subjects that require thorough documentation by Commvault.     1.) There is a long-standing recommendation to limit deduplication database disks to TWO PER MEDIA AGENT.’ I was able to track this recommendation as far back as Simpana 9.0, but I’m not sure if it pre-dates that.   This limit feels rather arbitrary, and it doesn’t take into account the performance capabilities of the host platform, or advances in computing kit since the original recommendation was made.   I can’t find any references showing WHY such a limit should exist.In my case, I have three Deduplication disks on my media agents (all NVMe SSD), and that runs with zero issues.  The Media Agents are spec’d over the recommended Extra Large spec, and they don’t even sweat.  I would like to issue a  call to Commvault to really explain why this limitation is in place. The Deduplication Building Block section of the documentation would be an ideal place for this information.  Th

9
S
Byte
2 years ago

Badge winners

Show all badges
Terms & ConditionsCookie settings