I'm trying to get both the HTTP code and the output of a curl command as part of a shell script, but in turn I'm trying to set both of them as variables. My code (which uses the pushbullet API) looks like this:
CURL_OUTPUT="$(exec 3>&1; \
HTTP_CODE="$(curl -s -S \
-w "%{http_code}" -o >(cat >&3) \
--header 'Authorization: Bearer '"$ACCT_TOKEN" \
-X POST https://api.pushbullet.com/v2/pushes \
--header 'Content-Type: application/json' \
--data-binary "$JSON")" \
)"
which should theoretically set $CURL_OUTPUT to the JSON returned by curl and $HTTP_CODE to the status code I get. Instead, I get only $CURL_OUTPUT; $HTTP_CODE is empty.
However, if I don't do the outer nest, like so:
exec 3>&1; \
HTTP_CODE="$(curl -s -S \
-w "%{http_code}" -o >(cat >&3) \
--header 'Authorization: Bearer '"$ACCT_TOKEN" \
-X POST https://api.pushbullet.com/v2/pushes \
--header 'Content-Type: application/json' \
--data-binary "$JSON")" \
The command works as expected; I get the JSON redirected to stdout, and the status code ends up in $HTTP_CODE.
So is it possible to get both outputs? Can I assign variables within a command substitution line?
Related
Using a postman as a base, I have a curl request here and I'm trying to return the access token.
AUTHORIZATION=$(curl --location --request POST 'https://some.url/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode "grant_type=$GRANT_TYPE" \
--data-urlencode "client_id=$CLIENT_ID" \
--data-urlencode "client_secret=$CLIENT_SECRET"\)
When I echo I get an output like:
{"access_token":"16WkRKbVpHWXlZekJsWVd...","token_type":"Bearer","expires_in":14400}
I want to extract the access_token and use in other parts of my script. I've tried the adding jq .access_token -r as seen below, but I'm just returning a null variable.
AUTHORIZATION=$(curl --location --request POST 'https://some.url/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode "grant_type=$GRANT_TYPE" \
--data-urlencode "client_id=$CLIENT_ID" \
--data-urlencode "client_secret=$CLIENT_SECRET"\
-s \
| jq .access_token -r)
Solutions here: extract token from curl result by shell script advise saving to file and grepping on it. I don't really want to save a token to a file if I can avoid it.
It looks like you flipped the parameter name and value when calling jq. I think it should be:
jq -r .access_token
not jq .access_token -r
Other than that your solution looks fine.
Step 1. Save the response in a variable
AUTHORIZATION=$(curl --location --request POST 'https://some.url/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode "grant_type=$GRANT_TYPE" \
--data-urlencode "client_id=$CLIENT_ID" \
--data-urlencode "client_secret=$CLIENT_SECRET"\)
Step 2 use jq.
AUTHORIZATION = `jq '.access_token' <<< "$AUTHORIZATION"`
Step 3 eliminate the quotes.
AUTHORIZATION=`echo "$AUTHORIZATION" | tr -d '"'`.
I am writing a bash script that does this :
Get a list of customers
For each customer, query an API (get files) This returns id and name
For each file, do a PUT.
My question is, I want to both "id" and "name" in my PUT body, so how can I get this in a single loop on "files" API. I have written this:
URL="https://some-url.com"
API_TOKEN="some-token"
get_customers() {
curl --insecure \
--fail \
--header "Accept: application/json" \
--header "Authorization: Bearer ${API_TOKEN}" \
-k ${URL}/api/org/customers
}
get_files() {
curl --insecure \
--fail \
--header "Accept: application/json" \
--header "Authorization: Bearer ${API_TOKEN}" \
-k ${URL}/api/files/files
}
put_capability() {
echo ""
curl --insecure \
--fail \
--silent \
--header "Content-Type: application/json" \
--header "Authorization: Bearer ${API_TOKEN}" \
--data #- \
${URL}/api/files/files/{id}
echo ""
}
customers=($(get_customers | jq --raw-output '.values[].id'))
for customer_id in "${customers[#]}"; do
fileshares=($(get_files ${customer_id} | jq --raw-output '.values[].id'))
fileshares-names=($(get_files ${customer_id} | jq --raw-output '.values[].name'))
for fileshare_id in "${fileshares[#]}"; do
#call function put_capability and do subsequent PUT.
#PUT needs two parameters, fileshares and fileshares-names
done
done
Basically, I don't want to call "get_files" twice as I am doing now. I need both the "fileshares" and "fileshares-names" in my subsequent PUT API. Can this be done in one for loop on file shares?
You can parse get_files output once by storing it in a temp file like that :
customers=($(get_customers | jq --raw-output '.values[].id'))
for customer_id in "${customers[#]}"; do
get_files ${customer_id} > /tmp/cust.txt
fileshares=($(cat /tmp/cust.txt | jq --raw-output '.values[].id'))
fileshares-names=($(cat /tmp/cust.txt | jq --raw-output '.values[].name'))
for fileshare_id in "${fileshares[#]}"; do
#call function put_capability and do subsequent PUT.
#PUT needs two parameters, fileshares and fileshares-names
done
done
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 4 years ago.
I am curling Azure Log Analytics for some info, but first I need to grab an OAuth token from 1 command and pass it into the next. I have the following Curl commands which I have tested fine on their own (copying pasting the output for the next input), however I want to pass the OAuth token output as a variable for an automation task, but for some reason it is not able to read the variable into the next command.
token=$(curl -X POST \
https://login.microsoftonline.com/{{subscriptionID}}/oauth2/token \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id={{clientID}}&client_secret={{clientSECRET}}&resource=https%3A%2F%2Fapi.loganalytics.io' \
| jq .access_token)
curl -X POST \
https://api.loganalytics.io/v1/workspaces/{{workspaceID}}/query \
-H 'Authorization: Bearer $token' \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{ "query": "AzureActivity | summarize count() by Category" }'
Unfortunately when I run this command it responds back that a token is needed.
{"error":{"message":"Valid authentication was not provided","code":"AuthorizationRequiredError"}}
However, if I were to echo the $token variable it shows that it was saved
beefcake#ubuntu:~$ echo $token
"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1...."
As I said, the commands work fine if I remove the token=$(..) and just copy/paste the output into the next input. Any ideas why this won't work for automation?
#Aserre had the right mindset. Turns out that jq copies the inverted commas " " from the string, whereas the bearer token requires none. Thus my first command should have looked like this:
token=$(curl -X POST \
https://login.microsoftonline.com/{{subscriptionID}}/oauth2/token \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id={{clientID}}&client_secret={{clientSECRET}}&resource=https%3A%2F%2Fapi.loganalytics.io' \
| jq -r .access_token)
Note the last line that has the -r command for removing the double quotes. Which showed an echo of:
beefcake#ubuntu:~$ echo $token
eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIs....
Note the " " removed from the echo. In addition to that, I had to alter the next command where I replaced 'Authorization: Bearer $token' with "Authorization: Bearer $token":
curl -X POST \
https://api.loganalytics.io/v1/workspaces/{{workspaceID}}/query \
-H "Authorization: Bearer $token" \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{ "query": "AzureActivity | summarize count() by Category" }'
I want to upload a big file with curl.
For that, I want to split it, without saving it to disk (like with split). I tried to use --continue-at with Content-Length.
curl -s \
--request PATCH \
--header "Content-Type: application/offset+octet-stream" \
--header "Content-Length: ${length}" \
--header "Upload-Offset: ${offset}" \
--continue-at "${offset}" \
--upload-file "${file}" \
"${dest}"
But curl "overshoots" and ignores Content-Length. Is there something like --stop-at? Alternatively, I have to use dd, if necessary.
EDIT
dd solution:
curl -s \
--request PATCH \
--header "Content-Type: application/offset+octet-stream" \
--header "Content-Length: ${length}" \
--header "Upload-Offset: ${offset}" \
--data-binary "#-" \
"${dest}" < <(dd if=${file} skip=${offset} count=${length} iflag=skip_bytes,count_bytes 2>/dev/null)
but if possible I would like to use only cURL..
If I run the following command, everything works great:
curl -s -o /dev/null -w %{http_code} \
http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
When I put this command into a variable in a /bin/sh script, and execute it, I get "HTTP 415 Unsupported Media Type" showing up in the server logs:
#!/bin/sh
read -d '' CMD << EOF
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
EOF
`$CMD`
I verified the command by printing it out and it looks example like the manual one I run:
#!/bin/sh
read -d '' CMD << EOF
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
EOF
echo $CMD
Output:
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com --header "Content-type: application/json" --request PUT --data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
For me it worked this way:
#!/bin/sh
read -d '' CMD << EOF
`/usr/bin/curl -s http://localhost:8080/someurl \
--header "Content-type: application/json"`
EOF
echo $CMD
What if evaluate it inside of EOF?