commvault.com commvault.com

Lack of a complete repository on Maintenance Releases

  • 1 February 2022
  • 8 replies
  • 395 views
T
Rank
Badge +1

Log4j showed me a huge flaw in they way Commvault pushes information related to Maintenance Releases (MRs).

 

Problem one is the software reading a list, allowing different installations to fetch different MRs as their ‘latest’ MR. I’ve got a colleague currently being offered 11.24.34 for example.

 

Problem two is ma.commvault.com not showing all releases or reasons why one got pulled. Documentation’s repo (https://documentation.commvault.com/11.24/expert/138863_list_of_maintenance_releases_for_feature_release_1124.html) isn’t complete either.
I had 11.24.31 offered at one point (through the software). This version was pulled, and only through support did I get to know the reason: the Server Event Manager crashing occasionally. This is a problem, because only 11.24.30 and later should include log4j 2.17, per its release notes: https://documentation.commvault.com/11.24/assets/service_pack/updates/11_24_31.htm. And I’ve now got two customers that have this problem, but at least they’re safe from log4j, so you win some, you lose some apparently…

 

Third problem is a combination. ma.commvault.com’s direct download links even have a 11.24.34 and 11.24.35 available when you change the URL (e.g. for WinX64), while only 11.24.32 should have gone public today, per support. Neither of those have release notes (same deal, by changing 11.24.29’s URL). While we’re at it, at the moment you don’t have a clear, publicly available MR that should protect from log4j fully.

 

As for the solution, as the single source of true information, I believe documentation.commvault.com should be kept up to date with work in progress entries for MRs, including MRs that got pulled and a reason why. (And while you’re at it, copy the download links for us lazy IT people…)

pgokhale 2 years ago

Yes, there were multiple packs released in last 6 weeks due to the developing situation around log4j.  Apache first released 2.16, then 2.17.  Then they found more stuff and released 2.17.1.  Being one of the fastest vendors to react, we were forced to follow the churn.

 

As of now:  just standardize everything on 11.24.34  (February 1st release) and all is good.    Release notes are referencing the same.  Next one is scheduled for March 1st 

 

Hope this helps.

View original
If you have a question or comment, please create a topic

8 replies

Ruben Renders
Rank
Userlevel 1
Badge

Hi Thomas!

You should update the customer environment to 11.24.29 as this includes the log4j fix you are looking for. More information can be found here: https://documentation.commvault.com/v11/essential/146231_security_vulnerability_and_reporting.html.

Hope this helps!

 

Onno van den Berg
Rank
Userlevel 7
Badge +19

Hi @Thomas-B,

We've had the same observation some months ago already and I agree with you that it is not ideal that Commvault decided to remove the weekly builds from the MA/Cloud portal. The ration behind is, is that due to the amount of releases is that they couldn't do extensive Q&A testing on the weekly builds. So they decided to release on a planned basis only once a month and only in the case of a ticket you would be able to get the intermediate releases. The log4j vulnerability itself was already covered as of the version as mentioned by @Ruben Renders and the reason that you still see reference is that they decided to upgrade all agents/components leveraging log4j (even though it is not using vulnerable) they decided to upgrade it anyway to remove/mitigate all the fuzz. 

Onno

L
Rank
Userlevel 6
Badge +15

Commvault decided to remove the weekly builds from the MA/Cloud portal.

This is something that (respectfully) makes me angry.

I’m the contact of my master account, I got no clear notification from Commvault stating this, whether by mail or even a pop-up while logging daily to MA.

I was expecting the weekly release of the hotfixpack by early january, but thought it had been delayed because of the huge impact log4j could have had on Commvault’s support. 

Here I understand that it’s not the root cause.

This is the kind of situation where I would complain about the expensive price paid (while I know most vaulters here are not involved in) :

When you buy the state of the art product, at a state of the art price, you expect state of the art services, which IMHO include such communication. 

Thanks to Onno for that information.

Onno van den Berg
Rank
Userlevel 7
Badge +19

@Laurent not sure why you are so upset about it because it has been in effect already for 6+ months. We ourselves noticed that the weekly rollup bundles sometimes also introduced unwanted side effects which again was a reason to focus on general release on a monthly basis with additional testing effort from Q&A to reduce this chance. 

Now what we all want I think is a strong reduction in the amount of defuncts that is found. This address you frustration at best and this accounts for all Commvault customers. Less defuncts → less tickets → less hotfixes → smaller/fewer weekly maintenance releases → less work → improve experience. In our case, if this is realized, than it would have a huge impact on our day-2 operations. I could spend much more time on POCs and expansion of services instead of troubleshooting errors and problems. 

When you buy the state of the art product, at a state of the art price, you expect state of the art services, which IMHO include such communication. 

In regards to the lack of open/clear communication I totally agree and I hope they improve on this. The community forum is one addition that could be used to share these messages. But you could see this change also from the angle that they want to improve the quality of the service. Now I think the bigger picture, that I touched previously, is way more important, because I would love to even see the monthly maintenance releases to be removed as that would indicate that the quality has gone up ;-) 

L
Rank
Userlevel 6
Badge +15

I get your point @Onno van den Berg . Sorry if I look that furious from my writings :smile:  

I’m in line with you.

Of course it’s better to improve the product, or make intensive tests before rolling out a release/hotfix pack.

I might have not noticed that it had been this way for that long, as I had in my mind the memory that it was almost a weekly release, while I got on (almost all) wednesdays a mail about the new hotfix pack available. Then I searched in my mailbox and found it was almost a monthly mail notification.

So, ouch, my bad :zipper_mouth:

 

As you mention it, the Community is a good channel in addition to the other means we have, to communicate and share knowledge,

I do appreciate the way we can share and update information here as we did for log4j, 

 

P
Rank
Userlevel 2
Badge +3

 

Maintenance releases are released monthly on First Tuesday of each month.  They appear in the gui,  MA site and software store.   This schedule has been in place for more than a year now.

 

Weekly packs are made for targeted use cases to solve Trouble tickets and should be used only as directed by support.  So far,  they were available to download via private links.  It is cumbersome to download and copy software.  so going forward, they will also be available in the GUI dropdown of third option of download software (java gui and command center)

 

Recently, this schedule was thrown off a bit due to emergency release due to log4j related fixes, but we are back on schedule as of Tuesday Feb 1st, 2022.

 

Hope this helps

T
Rank
Badge +1

Thanks for all the useful comments. I’m not sure we picked up on the monthly vs weekly MRs to be honest. :) Good to know and I’ll make sure to highlight that internally. MRs were more of a ‘look which version is available during an installation or update at a customer site and be done with it’ prior to Log4j.

What’s the rationale behind the release notes btw? Because they exist for MR29, 31 and 34. But not for MR32 (which support lead me to wait for two weeks ago). I’m not sure whether they were all monthlies, with 31 being a retracted monthly?

I’m guessing weeklies generally will end up in the next monthly’s release notes?

 

I got side tracked to using MR31 because its release notes indicated that it included Log4j 2.17. This was about when those multiple Log4j releases, well released, one after the other. It’s only after I created this post and reviewed this community and Mike’s Log4j topic (link), that I noticed the following:

CVE-2021-45105: Commvault can confirm that the affected Log4j2 function is NOT leveraged by Commvault software and thus there is no immediate need to update Commvault to use Log4j 2.17.

 

The original vulnerability post wasn’t as explicit. But I second Onno, MR34 is a good no-fuzz alternative going forward.


Lastly, it would still be nice to have the single list of all MRs (weekly & monthly) as a reference on documentation.commvault.com or here for partners. I don’t know how often an MR like MR31 introduces breaking behavior, but humans make mistakes, so it’s bound to happen again. :)

P
Rank
Userlevel 2
Badge +3

Yes, there were multiple packs released in last 6 weeks due to the developing situation around log4j.  Apache first released 2.16, then 2.17.  Then they found more stuff and released 2.17.1.  Being one of the fastest vendors to react, we were forced to follow the churn.

 

As of now:  just standardize everything on 11.24.34  (February 1st release) and all is good.    Release notes are referencing the same.  Next one is scheduled for March 1st 

 

Hope this helps.

Terms & ConditionsCookie settings