One of our auditors' controls is we provide a list of failed backups (from CommVault events) and they make a selection of several failed backups and ask to show that we have mitigated those failed backups. That's currently done by showing a successful rerun of the backup. We provide the details of the successful job from the log. Where we are stuck is, the auditors want to go back at least 12 months. We have the usual backups with a 30 day retention, some 6 months, some 12 months, some longer.https://documentation.commvault.com/commvault/v11/article?p=11952.htm is not entirely clear, but seems to say that the job history logs are kept as long as the protected data. Is the protected data and the job details correspondingly aged together and stored with the job on the MediaAgent’s Library? Or do the job logs reside somewhere else, and where is that?If the following are extended to 420 days (14 months), will that cause all of our backup protected data to also be retained 420 days (regardless of the 30 day or 6 month retention of the protected data in the storage policy)?Default settings:Days to keep the successful Backup Job Histories: Records are retained 7 days. Days to keep the failed/killed backup job and other job histories: Records are retained for 90 days. Days to keep Data Management and Collection Job Histories: Records are retained for 90 days.If the protected data retention is unaffected when I adjust the above settings to 420 days to keep the logs longer, what would be the impact (CommServer disk space, performance, etc.)?Then the source of the Event data for the auditors is the Event history, which is controlled by the setting (Java-GUI) CommCell > Control Panel > System > Threshold Values Event Retention Criteria: "Keep 99999 Events And All Events Less Than 7 Days Old". What is the impact if I adjust to "Keep 99999 Events And All Events Less Than 420 Days Old". Where are the resulting 420 days of events stored?Bottom line here is, can we safely increase these logging history settings, and what file system directory and/or SQL table do I need to monitor in order to avoid filling up a disk volume or destroying the performance of a database?Anyone else doing this?

Auditors want Logs for Events, Job History, Failed/Killed History, Restore History - 14 months worth

One of our auditors' controls is we provide a list of failed backups (from CommVault events) and they make a selection of several failed backups and ask to show that we have mitigated those failed backups. That's currently done by showing a successful rerun of the backup. We provide the details of the successful job from the log. Where we are stuck is, the auditors want to go back at least 12 months. We have the usual backups with a 30 day retention, some 6 months, some 12 months, some longer.

https://documentation.commvault.com/commvault/v11/article?p=11952.htm is not entirely clear, but seems to say that the job history logs are kept as long as the protected data. Is the protected data and the job details correspondingly aged together and stored with the job on the MediaAgent’s Library? Or do the job logs reside somewhere else, and where is that?

If the following are extended to 420 days (14 months), will that cause all of our backup protected data to also be retained 420 days (regardless of the 30 day or 6 month retention of the protected data in the storage policy)?

Default settings:

Days to keep the successful Backup Job Histories: Records are retained 7 days.
Days to keep the failed/killed backup job and other job histories: Records are retained for 90 days.
Days to keep Data Management and Collection Job Histories: Records are retained for 90 days.

If the protected data retention is unaffected when I adjust the above settings to 420 days to keep the logs longer, what would be the impact (CommServer disk space, performance, etc.)?

Then the source of the Event data for the auditors is the Event history, which is controlled by the setting (Java-GUI) CommCell > Control Panel > System > Threshold Values Event Retention Criteria: "Keep 99999 Events And All Events Less Than 7 Days Old".

What is the impact if I adjust to "Keep 99999 Events And All Events Less Than 420 Days Old". Where are the resulting 420 days of events stored?

Bottom line here is, can we safely increase these logging history settings, and what file system directory and/or SQL table do I need to monitor in order to avoid filling up a disk volume or destroying the performance of a database?

Anyone else doing this?

Page 1 / 1

Hi George,

Further details about those particula settings can be found in the following documentation link;
https://documentation.commvault.com/commvault/v11_sp20/article?p=11019.htm

In esence;

> The Backup Job History information is retained for as long as the job is within retention - Once the job has aged, we will start the countdown from the configured retention value.

> Backup job and other job histories applies to: Auxiliary Copy jobs, Stub Recall jobs, failed jobs, killed jobs, jobs that failed to start, and replication backup jobs - The countdown to prune the job history will start as soon as we finish the event.

> Data Management and Collection Job History will also start their countdown as soon as the associated job is finished.

The relevant information is stored within the CommServer Database under several tables - "JMBkpStats", "EvGuiAuditMessage" and "evMsg" to name a few. The best approach to understand whether the tables are still within limits or growing too much would be to regularly check the "Scale Statistic Report - Top 10 Largest Tables by Size" section, this will monitor all table sizes and flag any that may be under an outstanding growth.

https://documentation.commvault.com/commvault/v11_sp20/article?p=38751_1.htm

Since those tables are only used for monitoring/auditing purposes - They should not have any noticeable effect on the environment performance even if they grow, since we would normally only add new records into them and not perform any active searches during regular backup and restore operations.

Regards,
Javier

Thanks Javier.

To anyone else who have adjusted these default event and logging retention values, please respond with your experiences or any tips/tricks you’d like to share.

Hi @George , another similar setting to modify is the restore job history retention. “Days to keep the Restore job histories” is by default 90 days and due to audit requirements needs to be increased to a higher value, for instance 1 year. I have worked with some customer where several settings related with “required for compliance, auditing, or other reasons” have to be adjusted to meet their requirements. Consequence is database size grows and you need to monitor disk space on CommServe, on the DR export setting, etc..

Hello, I want to know if changing these settings will affect previous history records, or will it only take effect on new records?

@Shaw it affects all existing records as well as it only alters the purging parameters of the data aging process. In my opinion Commvault should alter the default and make sure information that affects data security are by default kept for a much longer period.

Does commvault recommend keeping the log to not exceed a particular size? or time range?, to lessen the impact on performance or the need to monitor for disk and db sizes? my organization requires a 8 year retention and I would be concerned that this would be detrimental to the commcerv.

Keep in mind using the CommServe as a data warehouse means an increase in size of the database and a decrease in performance. I would consider using a Private Metrics server for collecting and reporting on the extended job histories, and/or sending Alerts, Events, Audits, etc. to a third-party system like Splunk or a Syslog server.

https://documentation.commvault.com/2023e/essential/configuring_retention_settings_for_metrics_reports_data.html

https://documentation.commvault.com/2023e/expert/sending_alerts_audit_trails_events_and_logs_to_third_party_systems.html

Thanks,
Scott

Thanks Scott

we are looking at implementing Splunk but that project could be a while, our auditors are looking to hold 10 years of audit on the commserv. I was pretty sure that would be a problem with db size as you mentioned but I would need something authoritative from Commvault saying the recommendation shouldn't exceed x or something like that. I can’t find any documents that would support this.

Reply

Sign up

Login to the community

Scanning file for viruses.

This file cannot be downloaded