Skip to main content
commvault.com commvault.com
Solved

MA & VSA crashes when installed on RHEL 8.7 with Kernel version Linux 4.18.0-425.3.1.e18.x86_64

  • November 20, 2022
  • 9 replies
  • 1736 views
L
Byte
Forum|alt.badge.img+3

When MA and VSA binaries are installed on AWS instance of RHEL 8.7 with Kernel version Linux 4.18.0-425.3.1.e18.x86_64, both the instances crashes with error:  kernel: watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [rmmod:9167].

the message from OS shows:

cvblk: loading out-of-tree module taints kernel.
cvblk: module license 'CommVault Systems' taints kernel.
Disabling lock debugging due to kernel taint
cvblk: module verification failed: signature and/or required key missing - tainting kernel

 

As per Commvault documentation: https://documentation.commvault.com/2022e/expert/3515_block_level_backup_for_unix_system_requirements_01.html

the supported system requirement for block-level backup is RedHat Enterprise Linux 8.6 with kernel 4.18.0-372

 

Similar issue was reported earlier for the above kernel version, but now it seems to be supported.

Could you please update as when it will support 4.18.0-425 kernel version.



Note: The version 4.18.0-425 works for Commserve binaries, as the instance did not crash when only server components installed.
 

Thanks & Regards
Leena

 

Best answer by Marcel Geisen

Yes, we can confirm this behaviour. We already have a case open to check, because at the moment when RHEL machine boots up with Kernel  4.18.0-425.3.1.e18.x86_64   the server directly have high load and is not useable.

So before, when a new kernel for RHEL comes out it was booted and we only had the message that “cvblk” was not loaded and FREL was not available - but now with the machine is not responding anymore is a problem.

We are a service provider and most of our customers wants to use the newest kernel for security reason and audits.

So it would be helpful that we here get fast a support for this kernel or at least a solution that machines not get that high load.

Would be nice to hear here something from Commvault colleagues regarding this issue.

Thx a lot and BR.

Marcel

 

View original
    Did this answer your question?

    9 replies

    M
    Byte
    Forum|alt.badge.img+8

    Yes, we can confirm this behaviour. We already have a case open to check, because at the moment when RHEL machine boots up with Kernel  4.18.0-425.3.1.e18.x86_64   the server directly have high load and is not useable.

    So before, when a new kernel for RHEL comes out it was booted and we only had the message that “cvblk” was not loaded and FREL was not available - but now with the machine is not responding anymore is a problem.

    We are a service provider and most of our customers wants to use the newest kernel for security reason and audits.

    So it would be helpful that we here get fast a support for this kernel or at least a solution that machines not get that high load.

    Would be nice to hear here something from Commvault colleagues regarding this issue.

    Thx a lot and BR.

    Marcel

     

    L
    Byte
    Forum|alt.badge.img+3
    • Leena
    • Author
    • Byte
    • 7 replies
    • November 21, 2022

    @Mike Struening , not sure how the query is marked as best answer, but I am still looking for answer & support from commvault

    M
    1 person likes this
    • M
      Marcel Geisen
    L
    Vaulter
    Forum|alt.badge.img+14
    • LedoespVaulter
    • Vaulter
    • 249 replies
    • November 21, 2022

    As you highlighted, RHEL 8.7 is not listed as supported.

    I had similar issue when kernel was updated to 4.18.0-348 from 4.18.0-305 for RHEL 8.5 with 11.26.8

    At the end the solution was to update the binaries to 11.26.13.

     

    So you did well raising the incident with Support, before updating the kernel version please check the URL you mentioned earlier and if not listed there the kernel you want to install via Support case you can find out if new kernel version will be supported

    M
    Byte
    Forum|alt.badge.img+8
    Ledoesp wrote:

    As you highlighted, RHEL 8.7 is not listed as supported.

    I had similar issue when kernel was updated to 4.18.0-348 from 4.18.0-305 for RHEL 8.5 with 11.26.8

    At the end the solution was to update the binaries to 11.26.13.

     

    So you did well raising the incident with Support, before updating the kernel version please check the URL you mentioned earlier and if not listed there the kernel you want to install via Support case you can find out if new kernel version will be supported

    Hey, thanks for your answer. In general you are right, but in the past we often had the problem with new kernel, but then the modules were not loaded and the machine was not impacted. But this time the machine is after bootup with new kernel not useable and has impact to the machine production.

    We are Service Provider and our customers want for security reasons and audits often the newest kernel, and question is why these modules from commvault not made as dynamic modules which are created on runtime? Because this topic we have every half year with new kernel.

    @Ledoesp maybe you can ask in background why this is not possible otherwise it must be ensured, that if the kernel boots up and is not supported the module is not loaded so that the machine is not impacted.

    Thx & BR

    Marcel

    L
    Byte
    Forum|alt.badge.img+3
    • Leena
    • Author
    • Byte
    • 7 replies
    • November 22, 2022

    @Ledoesp - The same behavior is not observed on Commserve (linux) when updated to latest kernel. the impact is only on MA & VSA.

    M
    Byte
    Forum|alt.badge.img+8

    So got an answer from Dev Team in our Case:

     

    I had this checked with Development. Support for RHEL 8.7 is added on MR 11.28.35 which is likely to be published externally in early December.

    So for 11.28 it will supported with next MR in December for 11.26 is it not clear yet when it is implemented.

    Hope this helps a first feedback all waiting for this.

    Greets

    Marcel

    L
    1 person likes this
    • L
      Leena
    L
    Byte
    Forum|alt.badge.img+3
    • Leena
    • Author
    • Byte
    • 7 replies
    • November 23, 2022
    Marcel Geisen wrote:

    So got an answer from Dev Team in our Case:

     

    I had this checked with Development. Support for RHEL 8.7 is added on MR 11.28.35 which is likely to be published externally in early December.

    So for 11.28 it will supported with next MR in December for 11.26 is it not clear yet when it is implemented.

    Hope this helps a first feedback all waiting for this.

    Greets

    Marcel

     

    @Marcel Geisen Thank you for the update.

    O
    Bit
    Forum|alt.badge.img
    • Omkar
    • Bit
    • 1 reply
    • February 15, 2023

    We have the same issue with the RHEL 8.7 running on version 11.26.54 that is supposed to be configured as a FREL.

    We had opened a case with Commvault support and upgrading to 2022E was supposed to fix this issue.

    CV was upgraded to 2022E MR48 and the cvblk driver was still not loaded:

    Failed to load cvblk kernel module for machine Linux abc 4.18.0-425.3.1.el8.x86_64 #1 SMP Fri Sep 30 11:45:06 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux. This might be an unsupported kernel or the cvblk driver may be missing. Check Commvault documentation for supported kernel versions.

    S
    Byte
    Forum|alt.badge.img+6

    same issue here with rhel 8.7 and Commvault 11.24.86, FRELs and Index Servers dead since updating.

    Reply


    Most helpful members this week

    Terms & ConditionsCookie settings

    Cookie policy

    We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

     
    Cookie settings