Skip to main content
commvault.com commvault.com
Solved

Isilon NDMP Incremental - Selection rules?

B
Byte
Forum|alt.badge.img+4

Hello,

We have an Isilon cluster for primary storage for CIFS and NFS clients. We back it up to tape and disk libraries using 3-way ndmp. Incremental backups can be very large from time to time, so large that it seems like it might be backing up files that have not really changed. For example the current backup has transfered 1.7 million files. It seems unlikely that 1.7 million files have changed in the last 24 hours since the previous incremental (daily).

So the questions I have are -

  1. How are files selected for incremental backup and is there a way to modify/control it?
  2. Is there a way to get a report on what files were backed up in a given job and what changed in the file that caused it to be backed up?

Thanks,

Ron

Best answer by Harsh Desai

Hello bRonDoh,

When running backups using NDMP, there is no Commvault software on the NAS so it relies on the OneFS file system to keep a track of the changed data. By default, the OneFS file system walks each path to identify the changes and sends Commvault the data to back it up. When you have a large number of files to back up in a data set, even if only a few files were modified, regular file system incremental backups scan all the files to check for changes, which takes a large amount of time.

This is where the fast incremental options comes in. It leverages the Dell EMC Isilon/PowerScale Changelist API for backup. Changelist tracks the changes made to files between snapshots. When you run an incremental backup, only the files that changed are scanned, so the backup is faster.


You can have Commvault send the fast incremental flag to Isilon by following the doc below:
https://documentation.commvault.com/commvault/v11/article?p=19699.htm

With the above said though, in either case the files that are backed up by Commvault are selected by Isilon and not Commvault. As far as the report goes, you can run the backup job summary report and enable the option to include the protected objects to view a list of files.
https://documentation.commvault.com/commvault/v11/article?p=39616.htm

Hope this helps.
Harsh

View original
Did this answer your question?

9 replies

G
Vaulter
Forum|alt.badge.img+11
  • GseibakVaulter
  • Vaulter
  • 118 replies
  • May 7, 2021

Hi bRonDoh,

we should be using the snapshots to determine the differences because NDMP is a single DUMP of the file system and we don’t scan it. 

we rely on the snapshots to determine the diff.

https://documentation.commvault.com/commvault/v11/article?p=19699.htm

make sure the HW is handling the snapshots correctly?

 

Gary Seibak - TAM Commvault
Mike Struening
1 person likes this
  • Mike Struening
    Mike Struening
H
Vaulter
Forum|alt.badge.img+5
  • Harsh DesaiVaulter
  • Vaulter
  • 22 replies
  • Answer
  • May 7, 2021

Hello bRonDoh,

When running backups using NDMP, there is no Commvault software on the NAS so it relies on the OneFS file system to keep a track of the changed data. By default, the OneFS file system walks each path to identify the changes and sends Commvault the data to back it up. When you have a large number of files to back up in a data set, even if only a few files were modified, regular file system incremental backups scan all the files to check for changes, which takes a large amount of time.

This is where the fast incremental options comes in. It leverages the Dell EMC Isilon/PowerScale Changelist API for backup. Changelist tracks the changes made to files between snapshots. When you run an incremental backup, only the files that changed are scanned, so the backup is faster.


You can have Commvault send the fast incremental flag to Isilon by following the doc below:
https://documentation.commvault.com/commvault/v11/article?p=19699.htm

With the above said though, in either case the files that are backed up by Commvault are selected by Isilon and not Commvault. As far as the report goes, you can run the backup job summary report and enable the option to include the protected objects to view a list of files.
https://documentation.commvault.com/commvault/v11/article?p=39616.htm

Hope this helps.
Harsh

Mike Struening
1 person likes this
  • Mike Struening
    Mike Struening
B
Byte
Forum|alt.badge.img+4
  • bRonDoh
  • Author
  • Byte
  • 22 replies
  • May 7, 2021

@Harsh Desai Thanks for the feedback. We are using the Fast Incremental Backup option, so we do not have an issue with slow backups or long filescan times. I appreciate that Commvault relies on the Isilon to do the file selection. Will the “backup job summary report” indicate what changed for a file to have been backed up? IE will it indicate that the file is newly created, size changed, metadata/attribute changed, etc?

Cheers,

Ron

H
Vaulter
Forum|alt.badge.img+5

@bRonDoh Unfortunately the backup job summary report does not go into that much detail as it does not capture file change details. With file system backups (Linux and Windows), Commvault relies on either change journal or the last modified times to build a list of files that will be backed up. For NDMP, it relies on Isilon to compute a delta by comparing the current snapshot with the old snapshot. 

The backup job summary report will give you a summary like the backup times, throughput, size etc. The file information it will only include if the option is selected for it when running the report.

Regards

Harsh

B
Byte
Forum|alt.badge.img+4
  • bRonDoh
  • Author
  • Byte
  • 22 replies
  • May 14, 2021

@Harsh Desai Thanks for confirming that. I decided to try the File Level Analytics report and it does produce a list of individual files backed up and the modification timestamp. So that is a bit more helpful than the job summary report. Unfortunately it does not reveal why a file that has not been modified in the last 24 hours (ie since the last incremental) is being backed up. As you and others (and the documentation) have pointed out it is the Isilon system that tells the CommVault what to backup. So I will have to knock on the door of the Isilon support team to see why files that have not been modified are being backed up - and not just the “why” but also the means to control what is selected. Here is a summary table from the FLA Report of a recent job. Notice the huge number and size files backed up that have not been modified recently :

 

Distribution by Modification Time
Type Less than 3 Months 3-6 Months 6-12 Months 1-2 Years 2-5 Years More than 5 Years
Count 11500 72036 18568 80009 76746 77715

Total Size

 26.04 GB 57.31 GB 52.04 GB 281.12 GB 718.86 GB 1182.66 GB
Mike Struening
J
2 people like this
  • Mike Struening
    Mike Struening
  • J
    Johar Grewal
Mike Struening
Vaulter
Forum|alt.badge.img+23

@bRonDoh , if you find anything out, please come back and share your fidnings!

https://www.linkedin.com/in/michael-struening
Mike Struening
Vaulter
Forum|alt.badge.img+23

Hey @bRonDoh , hope all is well.  Gentle follow up to see if you got anything helpful from the Isilon support team to share.

Thanks!!

https://www.linkedin.com/in/michael-struening
B
Byte
Forum|alt.badge.img+4
  • bRonDoh
  • Author
  • Byte
  • 22 replies
  • June 25, 2021

Hi, All is well-ish. Gentle reply letting you know that nothing helpful from Isilon support team yet. They have kindly  informed me a few times as to how full/incremental works and linked me to their kb on the subject twice. In some cases a sub-dir has been renamed by an end-user and that causes all files/dirs below to be backed up. This is standard behaviour for all backups for as long as I have been in computing (35 years). So no real progress yet.

Cheers,

Ron

Mike Struening
Vaulter
Forum|alt.badge.img+23

Geez….I’m sorry they haven’t been very helpful.  Do they have any sort of monitoring feature to see what is accessing a file/folder a la procmon?

https://www.linkedin.com/in/michael-struening

Reply


Most helpful members this week

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings