Skip to main content

Is it possible to backup a directory and its subdirectories (several levels deep) and exclude all files, using a Windows File System agent, i.e. I only want to backup the directory structure.  I’ve looked at Filters and wildcards but could not see how to achieve this.

Yes there is 😃

I have done this for SQL servers where the SQL agent managed the DB's. Will have to check later today in the office, will get back to you. If no one answers sooner 🤣


I regret to say that my solutions is from a while back, this apparently is not possible anymore with easy wildcard solutions and in some situations not possible at all. My apologies for the inconvenience.

Have checked all combinations where I was able to filter files some sub levels, but not all.
I even tried to list all folders via cmd and enter these folders as separate filter lines, but this is not accepted by the java gui… For some reason the command center does accept these entries, but result is the same.

So concluding, either all subfolders and files are filtered or files are partially filtered.

You can only filter files from a specific folder, with the caveat that when a sub folder is defined while a higher level already is defined the filter for the sub folder wont work...

I think this is something which need to be re-evaluated by development…..


Hey @StephenT , thanks for the post!

Currently, there is no way to do this, and we’d need to submit a CMR.

Before that, though a key piece will be the use case.

What is your end goal for doing this?  Knowing the ‘why’ will help increase the likelihood of the CMR being accepted 🤓


Thanks for your prompt replies.
My end goal is to have the ability to do timely bare metal restores of computers that have large MS SQL Server databases.  These are all VMs.  Our backup schedules are: VM backups are done weekly, C drive file system backups daily and SQL Server database backups are done several times a day.  

All the system databases are on the C drive (which is a relatively small size drive).  Other databases and their log files are on the D and L drives (large drives).  It would be a waste of time (both during backup and restore) if the VM backup included the D & L drives as the databases on these drives would be out of date, so only the C drive is backed up during the VM backup.

The bare metal restore process is:
1) Restore the VM (C drive only) from the VM backup. As this is a small drive, this step does not take long. After the restore, the system databases are up and the other databases (on D and L ) are in a “Recovery Pending” state.

2) Manually attach the D and L disks (their properties are documented).

3) From a Commvault daily file system “directory-only” backup, re-create the required directories on the D and L drives. Most importantly, this creates the directories with the correct permissions.

4) Optional: Restore the C drive filesystem using the latest daily file system backup.  Other than the databases, there is little, or no state stored on the C drive so the last VM backup is likely to suffice.

5) Restore all the databases using the most recent SQL Server backup.

Alternatively, we could backup the D and L drives with a file system backup and filter out all the .dbf, .ldf, etc files.  This would also be capable of re-creating the directory structure. However, I’m trying to allow for the human factor and anticipate what could happen in future, e.g., if someone renames/copies mydb.dbf to mydb.dbf.25-apr-2022, it will not be filtered out.  To me, re-creating the directory-only structure from Commvault seems cleaner compared to restoring unwanted files that were created on these disks and left sitting around.


Appreciate the extremely detailed reply!

I reread it a few times to ensure I had the full request understood.

So essentially, you want to restore the folders (and permissions) on the DL and L: drives before you restore the databases (via the SQL iDA) to ensure the correct permissions are there, but you do NOT want the files within restored, because they are essentially database files you don’t want to be restored.

Is that the gist?  I want to get some SQL iDA folks to review to see if they have better ideas negating the need for the file permission side.


You could specify a single empty file to backup as the content on each folder. 

For example put a empty.txt file in any folder that has a db, maybe do it with with a pre-process script that just looks for empty mdf files and then drops a empty.txt file in there.

Then your content would be *empty.txt

Then do a permissions only restore which should recreate the folder structure.

Just spitballing.

Seems like a really quick solution though.


@StephenT 

I think your best bet here is Windows Files System backup for those other drives (or better yet just let default pick those up and have a different subclient for the C:\ drive in case any new drives are added it will pick them up.) Once that is setup use file filters for .mdf .ldf and .ndf to filter out all the MS SQL database files on that subclient.

The database files can not be renamed, if they are then its not attached or active database so I dont think that would be a concern. this will backup any other files that might have been placed on the drives filter the database files. You restore the entire drives/folders/permissions.


I appreciate all your replies.
@Mike Struening  – That is exactly the gist of what I meant.

Our current backup configuration is as @Scott Reynolds suggested (we use a Windows Files System backup and filter .mdf .ldf and .ndf files).  When I mentioned renaming the database files, I was trying to second guess what could happen in the future.  In this case the scenario is someone is restoring a database from the backup, but before they restore they keep the original database (by renaming the .mdf, ldf and .ndf files by adding a date file extension) until it is confirmed the restored database is working correctly and the original database is no longer required.

My original question was just me wondering “what if”. E.g.

  • Changes to the directory structure. The directory structure currently includes a “/MSSQL15.MSSQLSERVER/” directory.  What if one day the directory structure has a MSSQL16.MSSQLSERVER directory. Will someone remember to add this directory (as described by @christopherlecky ) to the content of the File System backup of the D and L drives using the “define content” method.
  • What if someone creates a non-database file on the D or L drives (for whatever reason). Using the “define filters” method, this file will be backed up unless the file extension is listed in the filters.

I have decided to continue with our current setup of defining filters to exclude .mdf .ldf and .ndf files. These filtered backups of the D and L drives are less than 2 MB.  One day it might backup a large non-database file that someone has created, but I think this is preferable to missing directories if the directory structure is ever changed and the changes aren’t added to the content of the File System subclient.

Thanks for everyone’s help.
 


Agreed on your assessment.  If you randomly grab a large database unexpectedly, you’ll notice via the backup size.