Commvault Webserver dot net Update

Hi @thomas.S 

Occasionally this can happen after an update.

Please try stopping all services, including IIS / World Wide Web Service.

Then start IIS / World Wide Web Service first and Commvault services second.

If there are any further issues after these steps, the next items for us to check are webservice.log and webconsole.log for any messages that might highlight the issue.

Thanks,

Stuart

Hi @Stuart Painter , 

I had sent the system into a reboot after installing the new dot net version.
Unfortunately I have already restored the snapshot. I will have to try again to look into the webconsole.log. 

kind regards

Thomas

Hi @thomas.S 

Thanks, webconsole.log will show messages for Web Console, Tomcat process that does rely on the web service to run, but if there’s an issue this might just show a connection/response failure.

The webserver.log however will show message for IIS that might indicate if there’s some issue with .NET.

Thanks,

Stuart

Hi @Stuart Painter, 

here I have the part of the log where the error messages starts to appier. Do you have a clue what would fix this issue ? 

  2 02/11 14:08:53    INFO  LogbackConfigBase:log:81 -     webconsole_DEFAULTLOGFILEBACKUPS: 5 (Integer, registry)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     webconsoleRequestWhiteList: null (String, default)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     webconsoleThemeOverride: false (Boolean, default)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     WebconsoleUseClientDLPSettings: false (Boolean, default)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     WebServerInstalled: {PKGNAME=Web Server, SINSTALLSTATE=Installed, SPRODUCTCODEGUID={E09B710D-8529-4B18-BB97-345FCB2DE5FB}} (Map, registry)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     whitelistedRedirectUrls: "" (String, default)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     WOPIWebServerUrl: null (String, default)
  2 02/11 14:08:53    INFO  LogbackConfigBase:?:? -     workflowRequestTimeout: 10 (Integer, default)
  2 02/11 14:08:53    INFO  ServerUtil:initializeApplication:133 - Initializing Application...
  1 02/11 14:08:53    WARN  ServerBroker:getResponseAndHandleExceptions:464 - Invalid response returned: 500
  1 02/11 14:08:53    ERROR DefaultHandler:sendPollRequest:249 - Web Service error: web service returned an unexpected response
 commvault.web.core.net.ServerException: INTERNAL_SERVER_ERROR
    at commvault.web.core.net.ServerBroker.getResponseAndHandleExceptions(ServerBroker.java:474)
    at commvault.web.core.net.ServerBroker.getServerResponse(ServerBroker.java:174)
    at commvault.web.core.net.ServerBroker.getServerResponse(ServerBroker.java:140)
    at commvault.web.core.net.ServerBroker.getResponse(ServerBroker.java:231)
    at commvault.web.core.net.ServerBroker$Proxy$_$$_WeldClientProxy.getResponse(Unknown Source)
    at commvault.web.core.WebServiceMonitor.sendPollRequest(WebServiceMonitor.java:219)
    at commvault.web.core.WebServiceMonitor.lambda$pollWebService$0(WebServiceMonitor.java:171)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
 1 02/11 14:08:53    ERROR DefaultHandler:sendPollRequest:263 - Web service status: RESPONSE_ERROR
  2 02/11 14:08:53    ERROR ServerUtil:initializeServerProperties:282 - Failed to load all server properties as an exception was thrown. 
 java.lang.Exception: Web service is not available: state=RESPONSE_ERROR
    at commvault.web.core.ServerUtil.initializeServerProperties(ServerUtil.java:242)
    at commvault.web.core.ServerUtil.initializeApplication(ServerUtil.java:140)
    at commvault.web.core.CVContextListener.contextInitialized(CVContextListener.java:40)
    at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4768)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5230)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Unknown Source)
 2 02/11 14:08:53    WARN  ServerUtil:?:? - Build timestamp not found in buildinfo. Defaulting to current timestamp
  2 02/11 14:08:53    INFO  ServerUtil:?:? - Application Initialized.
  2 02/11 14:08:53    INFO  ServerUtil:initializeApplication:150 - Initialize VMProv
  2 02/11 14:08:53    INFO  ServerUtil:initializeApplication:152 - Initializing show GUI options
  2 02/11 14:08:53    INFO  ServerUtil:initializeApplication:154 - Checking for Tomcat default app changes
  2 02/11 14:08:53    INFO  ServerUtil:?:? - Initializing hardware address
  2 02/11 14:08:53    INFO  CVContextListener:contextInitialized:42 - Context Initialized.
  2 02/11 14:08:54    INFO  AuthFilter:<init>:142 - Security Filter Started
  2 02/11 14:08:54    INFO  NegotiateSecurityFilter:<init>:75 - Negotiate Security Filter Loaded
  2 02/11 14:08:54    WARN  ServerBroker:getResponseAndHandleExceptions:464 - Invalid response returned: 500
  2 02/11 14:08:54    ERROR ServerUtil:?:? - Unable to load domain information. INTERNAL_SERVER_ERROR
  2 02/11 14:08:56    INFO  ServerUtil:ssoReport:554 - SSO report: SSO is NOT enabled because no domains are enabled for SSO on the CommServe; SecurityProtocol: [1/Negotiate]; CommServe SSO-enabled domains: <none>; Local FQDNs: [cvcstapp01.eos.lcl]; GetDomains response used to evaluate SSO status: [null]
  2 02/11 14:08:56    INFO  NegotiateSecurityFilter:?:? - SSO is disabled, Negotiate Security Filter is not initialized
  1 02/11 14:10:53    INFO  LogbackConfigBase:log:81 - Retrieving configuration properties using global params API
  1 02/11 14:10:54    INFO  LoginService:makeNewDummyLoginCred:164 - Performing dummy login for user- _+_PublicSharingUser_
  1 02/11 14:10:54    WARN  ServerBroker:getResponseAndHandleExceptions:464 - Invalid response returned: 500
  1 02/11 14:10:54    ERROR LogbackConfigBase:log:75 - Error while reading properties from global params (no similar errors will be logged unless the issue recurs after a successful read)
 commvault.web.core.net.ServerException: INTERNAL_SERVER_ERROR
    at commvault.web.core.net.ServerBroker.getResponseAndHandleExceptions(ServerBroker.java:474)
    at commvault.web.core.net.ServerBroker.getServerResponse(ServerBroker.java:174)
    at commvault.web.core.net.ServerBroker.getServerResponse(ServerBroker.java:140)
    at commvault.web.core.net.ServerBroker.getResponse(ServerBroker.java:231)
    at commvault.web.core.net.ServerBroker$Proxy$_$$_WeldClientProxy.getResponse(Unknown Source)
    at commvault.web.handlers.LoginService.makeNewDummyLoginCred(LoginService.java:166)
    at commvault.web.handlers.LoginService.doDummyLogin(LoginService.java:123)
    at commvault.web.handlers.LoginService.doDummyLogin(LoginService.java:112)
    at commvault.web.handlers.LoginService$Proxy$_$$_WeldClientProxy.doDummyLogin(Unknown Source)
    at commvault.web.core.ServerUtil.doDummyLogin(ServerUtil.java:1666)
    at commvault.web.core.ServerUtil.doDummyLogin(ServerUtil.java:1657)
    at commvault.web.core.ServerUtil.prepareRequestWithDummyLogin(ServerUtil.java:779)
    at commvault.web.config.CVConfigGlobalParamService.getGlobalConfigParams(CVConfigGlobalParamService.java:61)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at org.jboss.weld.bean.proxy.AbstractBeanInstance.invoke(AbstractBeanInstance.java:38)
    at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106)
    at commvault.web.config.CVConfigGlobalParamService$Proxy$_$$_WeldClientProxy.getGlobalConfigParams(Unknown Source)
Wrapped by: java.lang.Exception: Failed to retrieve global configuration properties using global params API
    at commvault.web.core.config.common.ConfigBase.readConfigValuesFromGlobalParams(ConfigBase.java:852)
    at commvault.web.core.config.common.ConfigBase.readConfigValues(ConfigBase.java:388)
    at commvault.web.core.config.common.ConfigBase.refreshConfigValues(ConfigBase.java:271)
    at commvault.web.config.CVConfig$Proxy$_$$_WeldClientProxy.refreshConfigValues(Unknown Source)
    at commvault.web.core.config.ConfigPoller$1.run(ConfigPoller.java:49)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
 

kind regards

Thomas

Hi Thomas,

I hope you are doing well.

Thanks for the log cuts, from the errors thrown it indeed indicates a problem with .Net Core, but it won't tell us exactly which one.

------------ 

ERROR DefaultHandler:sendPollRequest:249 - Web Service error: web service returned an unexpected response
 commvault.web.core.net.ServerException: INTERNAL_SERVER_ERROR
  1 02/11 14:10:54    WARN  ServerBroker:getResponseAndHandleExceptions:464 - Invalid response returned: 500

------------

Based on my experience with issue on other cases, here is what I would recommend you to do:

• Depending on which FR (Service Pack) you are running, check the Third Party Applications documentation and confirm the default version of .NET CORE we install with it: https://documentation.commvault.com/11.24/expert/121377_third_party_applications_installed_by_commvault_installer.html
• Download it from MS site by selecting just the link for Hosting Bundle: https://dotnet.microsoft.com/en-us/download/dotnet/3.1

• Clean any other versions of .NET CORE (not Framework) that could be installed.
• Use downloaded media to install or repair .Net Core. It should end up looking like this (Note that it has 3 components):

• Finally, restart IIS services by running iisreset command from cmd.

Note: For Service Packs before FR24, you can install any version within the 2.1.X series, but not 3.1.X. Same steps apply.

Please let us know if this helps resolving the issue. Good luck!

Hello @Sarahy , 

the update on our test environment did not work. The web/admin console did not work after replacing dot net. I have now opened a case here directly with Commvault. It will be investigated soon during a remote session to see what the problem is. 
That we must use the version from SP24 for our problem I have also heard. Our production environment is currently still running on SP20. So, as soon as the problem with the test environment is solved, we will update our production environment to SP24.

Kind regards

Thomas

@thomas.S , can you share the case number here?  I want top track its progress.

Thanks!

Hello @Mike Struening,

yes, of course.
Here’s the case number: 220211-321

Kind regards

Thomas

Hi @thomas.S 

Please be aware of the following kb article which may be related to your concerns as well.
https://kb.commvault.com/article/69161

Regards,
Sean

Hi @Sean Crifasi, 

Thanks for the tip. We have just installed dot net version 3.1.22. The web/admin console is running. I was not aware that only the dot net hosting needs to be installed. I had also installed the SDK, which probably led to this problem.

Regards

Thomas

@thomas.S , is everything working now as per @Sean Crifasi ‘s KB article?

I checked the incident and it is still open; wanted to be sure we catch everything up to each other 

I’ll take a simple solution any day!  thanks for sharing with us!