Hi bRonDoh,
we should be using the snapshots to determine the differences because NDMP is a single DUMP of the file system and we don’t scan it.
we rely on the snapshots to determine the diff.
https://documentation.commvault.com/commvault/v11/article?p=19699.htm
make sure the HW is handling the snapshots correctly?
Hello bRonDoh,
When running backups using NDMP, there is no Commvault software on the NAS so it relies on the OneFS file system to keep a track of the changed data. By default, the OneFS file system walks each path to identify the changes and sends Commvault the data to back it up. When you have a large number of files to back up in a data set, even if only a few files were modified, regular file system incremental backups scan all the files to check for changes, which takes a large amount of time.
This is where the fast incremental options comes in. It leverages the Dell EMC Isilon/PowerScale Changelist API for backup. Changelist tracks the changes made to files between snapshots. When you run an incremental backup, only the files that changed are scanned, so the backup is faster.
You can have Commvault send the fast incremental flag to Isilon by following the doc below:
https://documentation.commvault.com/commvault/v11/article?p=19699.htm
With the above said though, in either case the files that are backed up by Commvault are selected by Isilon and not Commvault. As far as the report goes, you can run the backup job summary report and enable the option to include the protected objects to view a list of files.
https://documentation.commvault.com/commvault/v11/article?p=39616.htm
Hope this helps.
Harsh
@Harsh Desai Thanks for the feedback. We are using the Fast Incremental Backup option, so we do not have an issue with slow backups or long filescan times. I appreciate that Commvault relies on the Isilon to do the file selection. Will the “backup job summary report” indicate what changed for a file to have been backed up? IE will it indicate that the file is newly created, size changed, metadata/attribute changed, etc?
Cheers,
Ron
@bRonDoh Unfortunately the backup job summary report does not go into that much detail as it does not capture file change details. With file system backups (Linux and Windows), Commvault relies on either change journal or the last modified times to build a list of files that will be backed up. For NDMP, it relies on Isilon to compute a delta by comparing the current snapshot with the old snapshot.
The backup job summary report will give you a summary like the backup times, throughput, size etc. The file information it will only include if the option is selected for it when running the report.
Regards
Harsh
@Harsh Desai Thanks for confirming that. I decided to try the File Level Analytics report and it does produce a list of individual files backed up and the modification timestamp. So that is a bit more helpful than the job summary report. Unfortunately it does not reveal why a file that has not been modified in the last 24 hours (ie since the last incremental) is being backed up. As you and others (and the documentation) have pointed out it is the Isilon system that tells the CommVault what to backup. So I will have to knock on the door of the Isilon support team to see why files that have not been modified are being backed up - and not just the “why” but also the means to control what is selected. Here is a summary table from the FLA Report of a recent job. Notice the huge number and size files backed up that have not been modified recently :
| Distribution by Modification Time |
| Type | Less than 3 Months | 3-6 Months | 6-12 Months | 1-2 Years | 2-5 Years | More than 5 Years |
| Count | 11500 | 72036 | 18568 | 80009 | 76746 | 77715 |
| Total Size | 26.04 GB | 57.31 GB | 52.04 GB | 281.12 GB | 718.86 GB | 1182.66 GB |
@bRonDoh , if you find anything out, please come back and share your fidnings!
Hey @bRonDoh , hope all is well. Gentle follow up to see if you got anything helpful from the Isilon support team to share.
Thanks!!
Hi, All is well-ish. Gentle reply letting you know that nothing helpful from Isilon support team yet. They have kindly informed me a few times as to how full/incremental works and linked me to their kb on the subject twice. In some cases a sub-dir has been renamed by an end-user and that causes all files/dirs below to be backed up. This is standard behaviour for all backups for as long as I have been in computing (35 years). So no real progress yet.
Cheers,
Ron
Geez….I’m sorry they haven’t been very helpful. Do they have any sort of monitoring feature to see what is accessing a file/folder a la procmon?