Solved

Did FR20 had API changes??

  • 12 February 2021
  • 8 replies
  • 186 views

Badge +2

We have a situation and/or I rather have a question in relations to FR20. After upgrading one of our commcells to FR20 with SP 32..half of our automation which is based on API calls started to fail and stopped working, meaning that some queries are passing through and API call works, but most of them do not work. I’ve tried to look through release notes of FR20 but did not find anything about such scenario and did not find anything in known issues of it.

 

Anyone aware of any similar cases where after upgrade, the API calls stopped working and have there been any changes to API itself, possibly it’s structure or how it is called?. I do have a suspicion that it might possibly be due to different versions of commcells? Because that upgraded comcell is impacting some queries to non-upgraded commcell as well.

Example:
 

Variable job_request is ‘http://comcell:81/SearchSvc/CVWebService.svc/ExecuteQCommand
so it sends command in variable data "command=qoperation execscript -sn QS_DataProtectionJobSummary -si @LastNDays='7'".

....

def get_commvault_clients(self):
commvault_clients = []
for api in self.api_url:
jobs_request = api['url'] + 'ExecuteQCommand'
jobs_response = requests.post(jobs_request, headers = api['headers'], timeout = 1800, data = "command=qoperation execscript -sn QS_DataProtectionJobSummary -si @LastNDays='7'")
if jobs_response.status_code == 200:
jobs = jobs_response.json()
for i in jobs['ExecScriptOutput']['FieldValue']:
if i['@Agent'] == 'Virtual Server':
vmname = i['@VM'].split("_")[0]
commvault_clients.append(vmname)
elif i['@Agent'] == 'Virtual Server' and i['@VM'] == 'N/A':
commvault_clients.append(i['@Client'])
else:
commvault_clients.append(i['@Client'])
return commvault_clients

So the above example will not return any output, except internal error 500 response from that API call when we are querying the commcell which has been upgraded to FR20. If I run this query against a commcell which is still at v11SP16 – it works.

 

icon

Best answer by Yash 12 February 2021, 18:14

There has been no change to the api. It looks like you’re missing a header. 

Could you please make sure you have this below header in your api[‘headers’] dictionary?

'Content-Type': 'application/x-www-form-urlencoded'

Just FYI, you could generate python code here and compare: https://api.commvault.com/#de96fee3-2f07-4601-a810-44baa761face

 

View original

8 replies

Userlevel 2
Badge +3

There has been no change to the api. It looks like you’re missing a header. 

Could you please make sure you have this below header in your api[‘headers’] dictionary?

'Content-Type': 'application/x-www-form-urlencoded'

Just FYI, you could generate python code here and compare: https://api.commvault.com/#de96fee3-2f07-4601-a810-44baa761face

 

Badge +2

There has been no change to the api. It looks like you’re missing a header. 

Could you please make sure you have this below header in your api[‘headers’] dictionary?

'Content-Type': 'application/x-www-form-urlencoded'

Just FYI, you could generate python code here and compare: https://api.commvault.com/#de96fee3-2f07-4601-a810-44baa761face

 

Hi Yash,

 

I have headers there, but my headers do not contain ‘Content-Type application/x-www-form-urlencoded’ as it was not nessecary and even now it works on v11SP16..

And as I have mentioned - this query works as expected to our commcell which is on v11SP16. Same query does not work to our commcell which we have upgraded from v11SP16 to v11SP20..

code snippet and it’s output:
...
            if response.status_code == 200:
                root = ET.fromstring(response.text)
                token = root.attrib['token']
                headers = {'Authtoken': token, 'Accept': 'application/json'}
            auth['headers'] = headers
            print(auth['headers'])

{'Authtoken': 'QSDK 36b67d5ebb59087143f29c9228e3a6b172823a610d34a2afc12ee2a64920d12f026252848d6703795ec8e81cd81d9313b537d4478a5d66304f22be23860d7bbcf2b4f2705c1e78741744bccb45b1fb7fDELETEDSOMELINESTOMAKETEXTSHORTER, 'Accept': 'application/json'}



I have added ‘Content-Type application/x-www-form-urlencoded’ to my headers and the query worked…

So question, with FR20 - some syntax became mandatory?

Badge

Hi Benjis, the header was always mandatory, but with FR20 we have moved to dotnet core framework with which the form data which you send would need this header mandatorily as the dotnet core framework makes sure that the Content-Type is a hard check and if it doesnt match, reply back with an exception without reaching our layer. 

Badge

Before upgrading to this SP I opened a case to make sure there was no API changes and the answer was that no changes occured.

So I upgraded, and we had API issues, just like the ones you mentionned.

Called back the support to troubleshoot, and same answer “no changes”.

But as Benjis, I could state it worked before that SP and failed to work after that SP, which was the only real change in our environment.

I understand dev teams that report that no change occured, but in fact, as explained there, non mandatory/structure checked things like headers have then changed. Leading to a mandatory code review on our side to be able to “do the same as before the SP upgrade”.

I’m not blaming anyone, I’m just willing to express a customer point of view. It is not that easy to find the proper and detailed way to code API codes. I would have appreciated to get this information as a potential issue/change when I open the case prior to the upgrade.

 

Userlevel 5
Badge +10

Hi @Laurent,

Appreciate the feedback, and thanks for sharing. We want to be laser-focused on removing these types of frustrations, and being a big organization means that things can occasionally fall through the cracks. Although not dropping $500m through the crack :open_mouth:

I think the scenarios are a little different - as in this case, it was always mandated (and documented) but not enforced until the product change came in that required it. That being said I can see how frustrating that could be if something was working and then post-upgrade there is an unexpected issue.

The general rule is that responses won't change between feature releases without being documented. If there is a change, then there is a breakage and should be treated as such. Change to REST have been documented in the past like in SP18;
 

For the GET Report REST API, in the License Summary response, the column title MaiboxName is changed to SMTP because SMTP is a more appropriate title for the data in this column.

 

In your case, the REST API response reportedly completely changed which was not communicated or noted in the documentation - that is an issue I will chase up internally, although I saw in the case (201014-187) that engineering checked internally and did not find a change, so something is still a little odd there on that one.

In either case, this will thread will bring a spotlight to these sorts of situations internally!

Userlevel 2
Badge +3

@Laurent  Yes, We understand your concern. We shouldn’t have allowed without sending the correct header in the first place. We’d try to avoid these going forward and update the release notes accordingly.

Badge +2

Oh, so we as a customer are not alone. Cool :)

 

Anyhow, apart from the point that headers now must have Content-Type we have found another changes in API’s. Some API’s had their responses changed after all.

 

For example this response output in SP16 returns ‘errorMessage’ value as either ‘Success’ or ‘Failed’, but with SP20 - this returns empty value only.

{'totalRecords': 1, 'pageNo': 0, 'errorMessage': '', 'errorCode': 0, 'pageSize': 2147483647, 'vmStatusInfoList':.....}

 

@Benjis We don't send success or failure in error Message even in Sp16. I tried quickly in the lab in Sp16, I see the same empty string. I hope you are referring to this API https://documentation.commvault.com/commvault/v11/article?p=125353_1.htm. Please let me know if you are referring to a different API

Reply