Solved

Siteminder fails to start because ClMgrs is using port 44442


Badge +2

Our security team are complaining that when they reboot their Siteminder servers, their process fails to start because ClMgrS process appears to be using port 44442.  Siteminder needs that port in order to start up.  What we end up having to do is to stop the commvault process in the client to let the Siteminder start; then restart commvault.

Does anyone know why Commvault need that port?   It is not in the list of standard ports that Commvault uses.  

https://documentation.commvault.com/v11/essential/7102_port_requirements_for_commvault.html#additional-ports-based-on-use-cases

 

Anyone know how to exclude that port by the CommVault agent?

icon

Best answer by Damian Andre 2 June 2022, 05:52

View original

6 replies

Userlevel 6
Badge +13

If you search under HKEY_LOCAL_MACHINE\SOFTWARE\CommVault Systems\Galaxy\Instance001 is there anything showing 4442?

Badge +2

Hi Aplynx, the Siteminder clients are Linux.  Do I check the CommServe for this key?

Userlevel 7
Badge +23

edit: I wrote this before I saw it was linux, but the same concept applies, except the registry is located in /etc/CommVaultRegistry/Galaxy/InstanceXXX

 

 

Technically, that port should be an ephemeral port - ie. dynamically allocated and should not be not statically bound by any application. Applications should know that that port could become unavailable the next time it wants to use it.

Commvault queries the OS and picks from the ephemeral or ‘dynamic’ port range that Microsoft specifies: 

netsh int ipv4 show dynamicport tcp

On my windows 2012 server, this is the output:

Protocol tcp Dynamic Port Range
---------------------------------
Start Port      : 33535
Number of Ports : 32000

This means that any application can lay claim to port 33535 and above for 32000 ports - i.e between 33535 and 65535. First come first serve!

For Linux the ephemeral port range is generally 32768–60999

As such siteminder is at fault for always requiring that port since it falls in the ephemeral / dynamic range - it should pick a port from outside of this range if it needs a static port that always has to be used (i.e that is why Commvault has 8400, 8403 etc.)

That being said, there is something you can do from our side.

When commvault starts up for the first time, it picks a bunch of dynamic ports to run its services on, it stores them in the registry under HKEY_LOCAL_MACHINE\SOFTWARE\CommVault Systems\Galaxy\Instance001\Runtime

For linux, it should be something like /etc/CommVaultRegistry/Galaxy/Instance001/Runtime/.properties

For this particular service, it ‘saves’ the last used port under SAVED_RecallService_v4

The next time the services run, it tries to reuse the port number listed there.

So all you really have to do is:

  1. stop the Commvault Client Manager Service or all Commvault services
  2. Start or restart siteminder so that it uses port 44442
  3. Delete the registry key “SAVED_RecallService_v4” in the above location
  4. Start the Commvault Client Manager Service or all Commvault services

This should then find a new port and use that going forward.

If you want to fix the port, so it always uses the same one, you can create a key called “FIXED_RecallService_v4” and set the decimal value to any available port (preferably outside the dynamic port range) - but that should not be necessary.

 

Hope this helps. 

Badge +2

@Damian Andre, Thank you so much.  I was able to find that profile, and it was exactly where you said it will be:
/etc/CommVaultRegistry/Galaxy/Instance001/Runtime/.properties

However port 44442 is not even being used.   When I reviewed the ClMgrS.log, it confirms port 43133 was the port used.  Not sure what is really conflicting with Siteminder now.  

[root[PROD]@omhqp14dae Log_Files]# cat ClMgrS.log | grep 5/31

---snip-------
27672 6c51 05/31 22:03:13 ### CCvNetworkServer::SelectData() - No port is specifed by user. Check if saved port can be reused
27672 6c51 05/31 22:03:13 ### CCvNetworkServer::SelectData() - Listening on port 43133/0
---snip-------

[root[PROD]@omhqp14dae Instance001]# cat Runtime/.properties
.cv.filler
CACHED_AppManager_network_access_v4 50123
CACHED_JobManager_v4 50226
CACHED_MediaManager_v4 50156
JobControlCVD_v4 45570
RecallService_v4 43133
SAVED_JobControlCVD_v4 45570
SAVED_RecallService_v4 43133
SAVED_SysMonEventSink_v4 41663
SysMonEventSink_v4 41663
argv0_0 /usr/openv/hds/Base/cvlaunchd
argv1_0 /usr/openv/hds/Base/cvd
argv2_0 /usr/openv/hds/Base/ClMgrS
argv3_0 /usr/openv/hds/Base/cvfwd
nCVDCCCPort 35928
nVALID 1
name0 cvlaunchd
name1 cvd
name2 ClMgrS
name3 cvfwd
path0 /usr/openv/hds/Base/cvlaunchd
path1 /usr/openv/hds/Base/cvd
path2 /usr/openv/hds/Base/ClMgrS
path3 /usr/openv/hds/Base/cvfwd
pid0 27670
pid1 27671
pid2 27672
pid3 27674
sid0 1
sid1 2
sid2 3
sid3 4
stime0 173897000000
stime1 173899000000
stime2 173900000000
stime3 173903000000

 

Userlevel 7
Badge +23

@jb-backup -in that case it sounds like it found the previous cached port in use the last time ClMgrS tried to start, and has already self-adjusted. You should be able to verify the process is running on the port specified by SAVED_RecallService_v4.

In that case, I think you should be good. Every service start should try to bind ClMgrS to 43133 going foward, which should not conflict with your other application.

In the event that there is still a conflict, try set FIXED_ RecallService_v4 to another port number.

 

I’m no linux guru by any means but you could try see if there is a conflicting process running on 44442 using:

sudo lsof -i -P -n | grep 44442

 

You can see though how siteminder is misbehaving - it should not be running a fixed port in the ephemeral range, as depending on the start order any application on the system could conflict with it. In future cases it could be another Commvault process too. I would definitely relay that to the security team and see if they can get it running on a port less than 32768 to get out of the dynamically assigned range.

Badge +2

  @Damian Andre Thank you so much for your help!

Reply