Solved

Description: Error occurred while processing chunk - error Code: [13:138]


Userlevel 4
Badge +12

Hi all!

could you advice me how to troubleshoot following types of error:

 

Error Code: [13:138] Description: Error occurred while processing chunk [xxx] in media [xxx], at the time of error in library [disklib01] and mount path [[xxx] /srv/commvault/disklib01/xxx], for storage policy [XXX] copy [Xxx] MediaAgent [svma1]: Backup Job [xxx]. Unable to setup the copy pipeline. Please check connectivity between Source MA [svma1] and Destination MA [svma1].

 

At a glance, it seems that it is not possible for CV to process chunk from the (index?)/disk library...However, the issue is connected with storage policy copy, that moves data from the disk library to the tape library (secondary copy). The main problem for us is that it is not possible to copy data to the tapes. Therefore, it may say Unable to setup the copy pipeline. The media agent is one server/device, that communicates with both disk and tape library. Lastly, the files in the related directories dont seem to be corrupted...

Any suggestions and ideas will be appreciated!

 

 

icon

Best answer by Mike Struening 2 September 2022, 21:09

View original

17 replies

Userlevel 7
Badge +23

@drPhil , based on the error, there’s a communication issue:

between Source MA [svma1] and Destination MA [svma1].

Can you check the connection reliability between those 2?

Userlevel 4
Badge +12

@drPhil , based on the error, there’s a communication issue:

between Source MA [svma1] and Destination MA [svma1].

Can you check the connection reliability between those 2?

Hi @Mike Struening, I hope you are doing well 😀

The thing is that there is only one Media Agent. Therefore, it looks a little bit incomprehensible.

Badge

What version of CV are you on? We started having these same errors after upgrading to 11.28.8.

We have the same error message intermittently occur where the media agent  can’t communicate with itself.

Unable to setup the copy pipeline. Please check connectivity between Source MA [ma004] and Destination MA [ma004].

We do have an active ticket with support.

Userlevel 7
Badge +23

@drPhil , I must be blind 🤣

Apologies for missing the obvious!

I concur with @RyanOJD .  If you are in the same boat version wise, create a support case.

If you can both share the incident number, I can track them.

Badge

@drPhil , I must be blind 🤣

Apologies for missing the obvious!

I concur with @RyanOJD .  If you are in the same boat version wise, create a support case.

If you can both share the incident number, I can track them.

Incident ID: 220712-540. 

Userlevel 7
Badge +23

Thank you!!

Badge +2

I’m seeing the same error messages as well.

Badge

After upgrading from 11.26.13 to 11.28.8 we began experiencing frequent errors on aux copy jobs, at times multiple errors per hour.

CV support escalated our support ticket to development. We had an existing additional setting on our HyperScale Media agents. The setting was sPipelineMode and it was set to AB:P

CV Development requested that we try removing this setting. After removing this setting the chunk errors with code 13:138 stopped occurring as did two other errors. The other two errors were timeout errors code 13:85 and unknown buffer type [0] received code 7:42. 

Userlevel 4
Badge +12

Hi all,

after investigating we have found out that there is an issue with the tape drive, that shows some errors on the management panel. I think, the error posted above (Error while processing chunk) is not related with the new version of Commvault since we have 11.24.48.

Our next step will be the most likely to obtain new tape drive. However, is it possible to change Global Secondary Policy Copy in the Policy Copy setting tab (it is grey out)? Now, we would need to use only one tape drive, but it is necessary to let the Policy Copy to choose different scratch pool...And Global Secondary Policy Copy can have only one scratch pool defined…..

Userlevel 3
Badge +9

Hello @drPhil 

You cannot re-associate a copy to a different Global Secondary Copy once its been associated.

You can put the issue drive into maintenance so that we won’t try to use it and you should be able to add the second scratch pool to the data paths of the Global Secondary Policy. 

Modifying the Properties of the Global Secondary Primary Copy Policy - https://documentation.commvault.com/11.26/expert/11653_modifying_properties_of_global_secondary_primary_copy_policy.html

 

Thank you,
Collin

Badge

Hello @Collin Harper 

We have seen this very error, “13:138 Error Occurred while processing chunk[...]”

[EDIT] Adding one more error message seen

“Failed to mount the disk media in library […] with mount path […] on mediaagent […]”

except this is during a job that backs up to an s3 bucket. 

Unfortunately I dont have access to the CV application myself, this is what I know of the problem and trying to see how can it be helped.

I dont know if there are tape libraries in the picture at the moment.

Is there anything you can point me to?

Thank you in advance!

Userlevel 1
Badge +5

Same issue happening now to me..it is for Tape library copy. Is the reason the drive need replace / faulty? is that a root cause (CRC error)? Error: Unable to setup the copy pipeline. Please check connectivity between Source MA [xxx-ma-001] and Destination MA [xxx-ma-001].

Userlevel 3
Badge +9

@AXVTKS 

Those errors “Failed to mount the disk media” are two part errors. There should be more to the error that states specifically what is wrong and why we failed to mount the storage resource. Can you post the full message you are receiving? Without this I wouldn’t be able to point you in a direction.

@Raj Balaraj 

Without logging we wouldn’t be able to say what the issue is. Can you see any errors in CvJobReplicatorODS.log on the source MediaAgent or CVD.log on the destination MediaAgent?

 

Thank you,
Collin

Userlevel 1
Badge +5
Error: Unable to send the stop data transfer control message to the tail. .

 

Userlevel 1
Badge +5
Failed to seek to the correct location on the media to start the data restore.

 

Userlevel 1
Badge +5

 

Userlevel 7
Badge +23

Sharing case resolution:

Solution:

Customer found after updating from 11.26.13 to 11.28.8 they were seeing numerous communications issues across multiple MAs. Issue was reviewed on session and escalated to Development. Development identified Additional Setting "sPipelineMode" was causing issue and recommended removing/disabling this. Once this was done Jobs started running without issue. Development continued to review for RCA but confirmed unable to determine without elevated logging and WireShark captures from time of issue. Discussed further with customer and confirmed case may be closed at this time as Jobs completing since removal of this additional setting.

Reply