A case should probably be opened. The usual issue is typically that one of the MAs has their IndexCache location filled up.
You can rearrange the order to the MongoDB Nodes, to make one or another the node the backup client. Note that if you are using LVM, you will need space in the Volume Group of the client used for the backup. You can check that with the command ‘vgdisplay’, you should have more than zero Free PE. Generally five to fifteen percent of the logical volume is recommended to be completely free.
--- Please check the space on all the Media Agents in the storage policy.
--- Verify the free space on all the MongoDB Nodes, and the Free PE amount on their volume groups.
Thanks, I'll probably have to open the case.
Hello, Community!
Anyone like this, maybe someone solved this problem, it appeared again. The media agents are fine.
Error Code: 14:109]
Description: Index processing failed on MediaAgent. This may be due to the index cache being full or because of a corrupt index.
Source: 5, Process: MongoDBCtrl
22855 5947 06/08 13:40:33 2108174 IDXCONNECT *****************USING NEW INDEXING LAYER**************
22855 5947 06/08 13:40:33 2108174 IDXCONNECT Stream Id not set, Generating it
22855 5947 06/08 13:40:33 2108174 IDXCONNECT Initializing Indexing connection to client [cmvt-1f-002] from [DBLE-1D-004], Setting log mode as offline
22855 5947 06/08 13:40:33 2108174 MongoDbIDA::MongoDbBackupCoordinator::createMongoDbMetadataInIndex(1312) - Replica set name=[rs0] dbPath=[/data] ArchFileId=[6234446]
22855 5947 06/08 13:40:33 2108174 MongoDbIDA::MongoDbUtil::SendMetadataAndFileHeader(4174) - Sending afile info for: [/rs0]
22855 5947 06/08 13:40:33 2108174 IDXCONNECT starting afile (2,113,6234446)
22855 5947 06/08 13:40:33 2108174 ERROR: CvFwClient::connect(): Connection failed on DBLE-1D-004: No incoming tunnel from cmvt-1f-002
22855 5947 06/08 13:40:33 2108174 CVOnDemandSvcClient::SubmitTask() - Unable to attach to cvd at cmvt-1f-002*cmvt-1f-002*8400 to launch on demand service.Error is 0xE8090068:{CCvNetwork::Attach_RemotePort(1400)} + {CCvNetwork::GetConnection(3273)} + {CSessionConnectionSocket::GetConnection(205)} + {CSessionConnectionSocket::ParseLastError(166)/EventMessage.0x09000068.9:104.(There is no incoming network connection from remote machine [cmvt-1f-002]. Please check if the remote machine is able to connect to the current machine using cvping tool on tunnel port)}
22855 5947 06/08 13:40:33 2108174 ERROR: CvFwClient::connect(): Connection failed on DBLE-1D-004: No incoming tunnel from cmvt-1f-002
22855 5947 06/08 13:40:33 2108174 CVOnDemandSvcClient::SubmitTask() - Unable to attach to cvd at cmvt-1f-002*cmvt-1f-002*8400 to launch on demand service.Error is 0xE8090068:{CCvNetwork::Attach_RemotePort(1400)} + {CCvNetwork::GetConnection(3273)} + {CSessionConnectionSocket::GetConnection(205)} + {CSessionConnectionSocket::ParseLastError(166)/EventMessage.0x09000068.9:104.(There is no incoming network connection from remote machine [cmvt-1f-002]. Please check if the remote machine is able to connect to the current machine using cvping tool on tunnel port)}
22855 5947 06/08 13:40:33 2108174 IDXCONNECT error at log commit message [0xE8090068:{I2::IdxLoggingClient::GotAfileInfo(489)} + {I2::IdxLoggingClient::GotAfileInfo(570)} + {I2::IdxLoggingClient::SendBeginStreamInfoToLM(210)} + {I2::NetworkUtils::OdsNetPoP::SendRecvMsg(231)} + {CCvNetwork::Attach_RemotePort(1400)} + {CCvNetwork::GetConnection(3273)} + {CSessionConnectionSocket::GetConnection(205)} + {CSessionConnectionSocket::ParseLastError(166)/EventMessage.0x09000068.9:104.(There is no incoming network connection from remote machine [cmvt-1f-002]. Please check if the remote machine is able to connect to the current machine using cvping tool on tunnel port)}]
22855 5947 06/08 13:40:33 2108174 MongoDbIDA::MongoDbUtil::SendMetadataAndFileHeader(4177) - Failed to send afile info
22855 5947 06/08 13:40:33 2108174 MongoDbIDA::MongoDbBackupCoordinator::createMongoDbMetadataInIndex(1373) - One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774), Failed to send afile info (W32.774): 0x80070306:{MongoDbIDA::MongoDbBackupCoordinator::createMongoDbMetadataInIndex(1373)/Failed to update send index metadata for repset [rs0]} + {MongoDbIDA::MongoDbUtil::SendMetadataAndFileHeader(4178)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Failed to send afile info}
22855 5947 06/08 13:40:33 2108174 IDXCONNECT destructor automatically calling backupDone for incomplete backup
22855 5947 06/08 13:40:33 2108174 ERROR: CvFwClient::connect(): Connection failed on DBLE-1D-004: No incoming tunnel from cmvt-1f-002
22855 5947 06/08 13:40:33 2108174 CVOnDemandSvcClient::SubmitTask() - Unable to attach to cvd at cmvt-1f-002*cmvt-1f-002*8400 to launch on demand service.Error is 0xE8090068:{CCvNetwork::Attach_RemotePort(1400)} + {CCvNetwork::GetConnection(3273)} + {CSessionConnectionSocket::GetConnection(205)} + {CSessionConnectionSocket::ParseLastError(166)/EventMessage.0x09000068.9:104.(There is no incoming network connection from remote machine [cmvt-1f-002]. Please check if the remote machine is able to connect to the current machine using cvping tool on tunnel port)}
22855 5947 06/08 13:40:33 2108174 IDXCONNECT error while finish logging [0xECCC0016:{I2::IdxLoggingClient::DoneLogging(930)} + {I2::IdxLoggingClient::DoneLoggingInt(909)/ErrNo.22.(Invalid argument)-Failed to get acknowledgment from Log Manager [0xE8090068:{I2::NetworkUtils::OdsNetPoP::SendRecvMsg(231)} + {CCvNetwork::Attach_RemotePort(1400)} + {CCvNetwork::GetConnection(3273)} + {CSessionConnectionSocket::GetConnection(205)} + {CSessionConnectionSocket::ParseLastError(166)/EventMessage.0x09000068.9:104.(There is no incoming network connection from remote machine [cmvt-1f-002]. Please check if the remote machine is able to connect to the current machine using cvping tool on tunnel port)}]}]
22855 5947 06/08 13:40:33 2108174 MongoDbIDA::MongoDbBackupCoordinator::OnComplete(1063) - Failed to create MongoDB index metadata [0x80070306:{MongoDbIDA::MongoDbBackupCoordinator::createMongoDbMetadataInIndex(1373)/Failed to update send index metadata for repset [rs0]} + {MongoDbIDA::MongoDbUtil::SendMetadataAndFileHeader(4178)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Failed to send afile info}].. Failing the backup
22855 5947 06/08 13:40:33 2108174 DistributedIDA::CMaster::endPhaseTasks(4073) - 0x80070306:{MongoDbIDA::MongoDbBackupCoordinator::OnComplete(1064)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Failed to create MongoDB index metadata [0x80070306:{MongoDbIDA::MongoDbBackupCoordinator::createMongoDbMetadataInIndex(1373)/Failed to update send index metadata for repset [rs0]} + {MongoDbIDA::MongoDbUtil::SendMetadataAndFileHeader(4178)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Failed to send afile info}].. Failing the backup}
22855 5947 06/08 13:40:33 2108174 DistributedIDA::CMaster::endPhaseTasks(4091) - m_jsStatus:2 m_jsStatusFor:1 m_jsPendingCause:4
22855 5947 06/08 13:40:33 2108174 BKP CALLED COMPLETE (PHASE Status::FAIL), 2108174. Token [2108174:5:1:0:0:5411]
22855 5947 06/08 13:40:33 2108174 main(172) - ---------------------
22855 5947 06/08 13:40:33 2108174 main(173) - ENDING DistributedIDA
22855 5947 06/08 13:40:33 2108174 main(174) - =====================
Look, what's the problem I don't understand?
The problem seemed to be resolved, it was due to the fact that the MA and the Mongo Host did not resolve each other using the host name.