commvault.com commvault.com
Solved

MongoDB Scheduled Full Backup goes pending state

  • 6 September 2021
  • 3 replies
  • 425 views
ibrahimemrekaya
Rank
Userlevel 3
Badge +7

Hi,

Our customer has MongoDB systems. Versions;

Commvault version: 11.20.64

MongoDB version: 4.4.4

OS version: ubuntu 20.04

I added MongoDB pseudo clients successfully. I took FULL backup manually from subclient. I created schedules for FULL and LOG backups.

Oplog Schedule

If I take full backup from subclient manually and run log backup from schedule, i took error like;

Error Code: [7:247] Description: Failed to backup MongoDB oplog dumps. Submit FULL SNAP backup immediately to resync the oplog dumps

I try to take full backup from schedule and i took error on archive index phase;

Please check the log files for this job for more details.

I checked logs and i saw this error in MA’s archiveIndex.log;

26120 44e8  09/06 09:18:41 10338926 Log Directory [D:\Commvault\IndexCache\CvIdxLogs\2\8F261E36-5EA5-4976-A70B-1D3FDD6F6F2E\J10338926_snap_skipCatalog] does not exist!, Setting Hard Indexing Failure

I couldn’t find anything for this error. Anyone can help me? Thank you.

Best regards,

İbrahim

icon

Best answer by Stuart Painter 6 September 2021, 10:48

View original
Best Regards, İbrahim

3 replies

S
Rank
Userlevel 7
Badge +15

Hi @ibrahimemrekaya 

Thank you for the question.

There was a historic issue with MongoDB that caused the 2nd error you are seeing, however internal notes suggest the fix for this issue was implemented in SP17 and I see you are running SP20, but it is worth checking to be sure.

26120 44e8  09/06 09:18:41 10338926 Log Directory [D:\Commvault\IndexCache\CvIdxLogs\2\8F261E36-5EA5-4976-A70B-1D3FDD6F6F2E\J10338926_snap_skipCatalog] does not exist!, Setting Hard Indexing Failure

The issue is due to the fact that the 'Skip Catalog Phase for Intellisnap Operations' option is turned on by default for MongoDB schedule policy tasks (when, not only should it be off, the option should not even be present in the GUI):

Please ensure this option is disabled and re test backups.

Please also check mongodbctrl.log on the secondary node to see if anything else is highlighted there.

Thanks,

Stuart

Customer Support
ibrahimemrekaya
Rank
Userlevel 3
Badge +7

Hi @Stuart Painter ,

Thank you for reply. I can take full backup from schedule now. But i tried to oplog backup from schedule, it failed;

Error Code: [7:247]
Description: Failed to backup MongoDB oplog dumps. Submit FULL SNAP backup immediately to resync the oplog dumps

What is the best practice for MongoDB backup schedules? Customer wants to take full backups every day and take log backups every hour. Documentation says create auto schedule for oplog;

https://documentation.commvault.com/commvault/v11/article?p=108343.htm 

How can i manage this? Is this configuration correct?

 

 

Best Regards, İbrahim
ibrahimemrekaya
Rank
Userlevel 3
Badge +7

Hi,

I saw errors in mongodbctrl.log;

778548 be134 09/08 09:23:22 10351375 MongoDbIDA::MongoDbUtil::RunMongoCmd(1218) - Error during execution of cmd [mongodump --port 27017 --host 10.115.208.80 --db local --collection oplog.rs --out "/opt/commvault/iDataAgent/jobResults/2/4947"  --query '{"ts":{"$gt":{"$timestamp": {"t": 1631081250, "i": 1}}},"ns":{"$nin":["config.system.sessions","config.transactions","config.transaction_coordinators"]}}'] rc [127] out [/bin/sh: 1: /usr/bin/mongodump: not found]

 

778548 be134 09/08 09:23:22 10351375 MongoDbIDA::MongoDbUtil::RunMongoDBDump(2670) - Failed to create oplog dump [/bin/sh: 1: /usr/bin/mongodump: not found]

 

I found this conversation;

Note: Starting from MongoDB 4.4, the mongodump and mongorestore utilities must be downloaded separately and copied to the bin path.

 

Thank you for your help.

Best Regards, İbrahim

Reply


Terms & ConditionsCookie settings