How to capture output of curl piped with grep into a variable in a shell script - bash

Question

I realise there are quite a few posts that show how to capture output of curl or grep, but haven't been able to find any post where I can capture the value of a group in the grep's regular expression.
I need to capture output of the following command in a variable inside a shell script.
I am working on API calls of Qlik tool. The way the APIs work, is that you first have to call the login operation, which returns a session key and then use this session key in the subsequent API calls, like this
https://help.qlik.com/en-US/enterprise-manager/November2021/Content/EnterpriseManager/EnterpriseManager_APIGuide/CurlAPI/api_login.htm
cURL example
Active Directory request
curl -i -k --header "Authorization: Basic cWFAcWE6cWE=" https://computer.network.net/attunityenterprisemanager/api/v1/login
Response
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/html
Server: Microsoft-HTTPAPI/2.0
EnterpriseManager.APISessionID: J3cKzWIbi_w6Fr1G-tO03Q
Date: Mon, 26 Dec 2016 17:02:01 GMT
And then use the EnterpriseManager.APISessionID
https://help.qlik.com/en-US/enterprise-manager/November2021/Content/EnterpriseManager/EnterpriseManager_APIGuide/CurlAPI/api_aem_getserver.htm
So, I am trying to get this EnterpriseManager.APISessionID value in a variable, so I can use it in the script to call other operations.
If I run the below command on the command line it prints the value on the terminal,
curl -i -k --header "Authorization: Basic mybase64idpwvalue" https://qlik-qem-dev.abc.com/attunityenterprisemanager/api/v1/login | grep '(EnterpriseManager.APISessionID: )(.*)' | echo "${BASH_REMATCH[2]}"
This is the output on command line
eu_3qrEWJFAz_4hL7bOIvA
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
but if I put it in a .sh file and execute it, then the variable value is empty
API_SESSION_ID=$(curl -i -k --header "Authorization: Basic mybase64idpwvalue" https://qlik-qem-dev.abc.com/attunityenterprisemanager/api/v1/login | grep 'EnterpriseManager.APISessionID: ')
API_SESSION_ID2=$(curl -i -k --header "Authorization: Basic mybase64idpwvalue" https://qlik-qem-dev.abc.com/attunityenterprisemanager/api/v1/login | grep '(EnterpriseManager.APISessionID: )(.*)')
#API_SESSION_ID=$(curl -i -k --header "Authorization: Basic mybase64idpwvalue" https://qlik-qem-dev.abc.com/attunityenterprisemanager/api/v1/login) | grep '(EnterpriseManager.APISessionID: )(.*)' | echo "${BASH_REMATCH[2]}"
echo "API_SESSION_ID : ${API_SESSION_ID}"
echo "API_SESSION_ID2 : ${API_SESSION_ID2}"
I tried some other combinations of the command in the script, but am unable to capture the value in a variable. It is always empty
How can I get that populated?
Thanks

Answer 1

First, I don't know where eu_3qrEWJFAz_4hL7bOIvA comes from in your example. I'll assume you meant J3cKzWIbi_w6Fr1G-tO03Q.
Second, your command line output doesn't make sense, unless you played around with the shell and some global state remains. When I run it, I get a empty output, which is what I expect.
Third, BASH_REMATCH is a bash variable that captures the result of regex matching from bash operators ([[ string =~ regex ]]). It's not aware of grep, perl, sed, or any other programs using regex.
That's why when I run your command I get no output - BASH_REMATCH is empty.
Last, your grep expression uses unquoted parentheses, and you didn't pass -E, so it looks for parentheses in the text, and it wouldn't find them. So even if BATCH_REMATCH was aware of grep, grep would still return an empty result.
All that said, sed does support capture:
API_SESSION_ID="$( \
curl -i -k \
--header "Authorization: Basic mybase64idpwvalue" \
https://qlik-qem-dev.abc.com/attunityenterprisemanager/api/v1/login \
| sed -n 's/\(EnterpriseManager.APISessionID: \)\(.*\)/\2/p' \
)"
echo "API_SESSION_ID : ${API_SESSION_ID}"
That \2 represents the second capture group. But you don't actually need the first:
API_SESSION_ID="$( \
curl -i -k \
--header "Authorization: Basic mybase64idpwvalue" \
https://qlik-qem-dev.abc.com/attunityenterprisemanager/api/v1/login \
| sed -n 's/EnterpriseManager.APISessionID: \(.*\)/\1/p' \
)"
echo "API_SESSION_ID : ${API_SESSION_ID}"

Answer 2

BASH_REMATCH isn't related to grep. It must be called after regex matching.
GNU grep (perl positive look behind):
echo "$example_header" |
grep -Po '(?<=^EnterpriseManager\.APISessionID: ).*'
Pure bash:
[[ $example_header =~ (^|$'\n')(EnterpriseManager\.APISessionID: )([^$'\n']*) ]]
echo "${BASH_REMATCH[3]}"
Both print:
J3cKzWIbi_w6Fr1G-tO03Q
For input:
example_header=\
'Response
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/html
Server: Microsoft-HTTPAPI/2.0
EnterpriseManager.APISessionID: J3cKzWIbi_w6Fr1G-tO03Q
Date: Mon, 26 Dec 2016 17:02:01 GMT'
# Or: example_header=$(curl -i -k --header "Authorization: Basic cWFAcWE6cWE=" https://computer.network.net/attunityenterprisemanager/api/v1/login)
Don't forget to escape . and to match the start of the line (to avoid matching NotEnterpriseManager...).