Variable expansion in shell script - shell

I am having some problem expanding a variable.
First I set the value of the key.
KEY=Basic YWRtaW46YWRtaW4=
Then I tried calling it in curl but it was not working. To diagnose the problem, I decided to print the actual command that was being run. This was the result
print curl -H `print "Authorization: " $KEY` "http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=0"
>> http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=0
It was only printing the string after the KEY. Whereas, If I replaced it myself, it was printing the correct output.
print curl -H "Authorization: Basic YWRtaW46YWRtaW4=" "http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=0"
>> curl -H Authorization: Basic YWRtaW46YWRtaW4= http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=0


You need to enclose the value in quotes because you have a space.
KEY="Basic YWRtaW46YWRtaW4="
Doing KEY=Basic YWRtaW46YWRtaW4= without quotes is like assigning "Basic" to "KEY" and nothing to another variable called "YWRtaW46YWRtaW4".
Then simply use it like so:
curl -H "Authorization: $KEY" "http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=0"

Related

curl set dynamic header token throws invalid key

Im trying to hit a rest api with token in header.
apikeyName="$(date '+%s')"
key=$(curl -k -X POST -H "Content-Type: application/json" \
-d '{"name":"'$apikeyName'", "role": "Admin"}' \
http://admin:admin#localhost:3000/api/auth/keys | jq '.key')
echo $key
# # Alerting API
curl -k -X GET 'http://localhost:3000/api/alert-notifications' -H 'Authorization: Bearer '$key'';
Terminal output
"eyJrIjoiaWJPaDFFZXZMeW1RYU90NUR4d014T3hYUmR6NDVUckoiLCJuIjoiMTY3NTM1OTc4OCIsImlkIjoxfQ=="
{"message":"invalid API key","traceID":""}
First 1 is the key printing and last one from api response. I tried to hardcode the key and it works.

Short answer: Use jq -r '.key' to extract the key from the json response without adding quotes to it.
Long answer: There is a difference between quotes on the command line and quotes embedded in a variable. Consider:
key='"abcd"'
printf '%s\n' $key "abcd"
# prints:
# "abcd"
# abcd
Quotes on the command line are bash syntax. Bash notes what is being quoted and then removes the quotes from the command line when it's done, thus printf only prints abcd in the second case above.
Quotes inside a variable are plain old data. Bash doesn't do anything with them, so they get passed through to the command like any other data and printf prints "abcd" in the first case.
In your curl case the receiver doesn't expect the key to have quotes embedded in the data. So, curl -blah "keydata" works fine because bash takes the quotes out, but curl -blah $key fails because bash does NOT remove the embedded quotes.
See also: BashParser

Extract a specific value from an output stream in bash

I have this command:
`curl -X POST https://server.domain.com/v2/jobs/$job_ID/runs?project_id=$project_ID -H "Content-Type: application/json" -H "Authorization: Bearer ${token}" -d "{ \"job_run\": {} }"
and it gives this output
quote
Response from job execution is {"metadata":{"rov":{"mode":0,"collaborator_ids":{}},"project_id":"aff59748-260a-476e-9578-b4f4a93e7a92","sandbox_id":"aff59748-260a-476e-9578-b4f4a93e7a92","usage":{"last_updated_at":"2021-02-12T00:31:19Z","last_updater_id":"1000331040","last_update_time":1613089879840,"last_accessed_at":"2021-02-12T00:31:19Z","last_access_time":1613089879840,"last_accessor_id":"1000331040","access_count":0},"name":"Notebook Job","description":"","tags":[],"asset_type":"job_run","origin_country":"us","rating":0,"total_ratings":0,"catalog_id":"d47cd45c-0161-4b89-ba4f-c0e48272f08e","created":1613089879840,"created_at":"2021-02-12T00:31:19Z","owner_id":"1000331040","size":0,"version":2,"asset_state":"available","asset_attributes":["job_run"],"asset_id":"5d9dae4e-0dfe-4e53-9e56-18a4bb44af","asset_category":"USER"},"entity":{"job_run":{"job_ref":"28723316-9373-44ba-9229-7c796f21b099","job_name":"Dummy Job for E2E Testing","job_type":"notebook","state":"Starting","isScheduledRun":false,"configuration":{"env_id":"jupconda37-aff59748-260a-476e-9578-b4f4a93e7a92","version":"ca0b9a58-ba0d-4d28-93d5-3cdc66a5b137","env_type":"notebook","env_variables":[],"job_manager_id":"9268a8a8-72e4-47a1-a851-65f38d8284ce","notebook_job_output":{},"command_line_arguments":[]},"project_name":"Tahoe Migration"}},"href":"/v2/assets/5d9dae4e-0dfe-4e53-9e56-ee18a4bb44af?project_id=aff59748-260a-476e-9578-b4f4a93e7a92"}
I want to extract the value of asset_id from the output stream and store it in a variable within shell script. I tried storing the response in a bash variable and tried to parse with awk, it didn't work. Could one of you please provide me an idea as to how to go about this?
Thank you

jq is much better for processing json, although you need to strip off the "quote Response from job execution is " prefix first. Something like this:
response=$(curl ...)
asset_id=$(echo "${response#*Response from job execution is }" | jq -r '.metadata.asset_id')

If that's the only thing you need, you can solve it with sed:
<YOUR_CURL> | sed 's/.*"asset_id":"\([0-9a-z\-]*\)".*/\1/'

Pass variable in curl to send message to slack [duplicate]

This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 2 years ago.
I'm trying to send a message to slack using the curl command. I wanted to use a variable in the curl command
curl -X POST -H 'Content-type: application/json' --data '{"text": $text }' "https://hooks.slack.com/services/blabla"
here the $text refers to the text I want to send.
this is how $text looks like
text="The following jobs are in queue: $jobs"
If you try echo'ing thext, it will be something like
echo $text
The following jobs are in queue: "job1" "job2"
But the above curl command is giving invalidpaylod

You have the text variable in single quotes and so the content won't be expanded. Put single quotes around the variable:
curl -X POST -H 'Content-type: application/json' --data '{"text": '$text' }' "https://hooks.slack.com/services/blabla"

Variable expansion is not working as intended with curl

I am having some trouble figuring out the expansion of a variable in my shell script. If I replace the variable with the desired string it works.
#!/bin/zsh
KEY="$(curl -Ivs -X GET "http://admin:admin#192.168.1.1" &> >(awk '/^> Authorization/{ print $3 " " $4 }'))"
# The string returned by the curl and awk command is Basic "YWRtaW46YWRtaW4=" without double quotes.
curl -H "Authorization: $KEY" "http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=1"
# This doesn't work
curl -H "Authorization: Basic YWRtaW46YWRtaW4=" "http://192.168.1.1/userRpm/WlanMacFilterRpm.htm?Page=1&exclusive=1"
# This works
The only thing thats different in the above two lines is.
-H "Authorization: $KEY"
-H "Authorization: Basic YWRtaW46YWRtaW4="

HTTP generally uses DOS style CR+LF line terminators. Whenever you parse data out of curl, you have to account for this.
To check if this is the problem, run your script with bash -x yourscript or zsh -x yourscript to see trace output that shows otherwise invisible carriage returns:
var=$'value\C-M' # zsh
var=$'value\r' # bash
(Dash and ash/busybox unfortunately doesn't highlight this problem, so try with one of the above shells)
To strip them, pipe your data through tr -d '\r'.

bash curl POST request: "-d" option using a variable

I am trying to make a POST request. The following works without any problem;
# Get expected response
curl http://localhost:9020/xxx/xxxx/ -H Content-type:application/json
-H Accept:application/json -d '{"LED":{
"language":"CHINESE"}, "text":"1928年11月22日"}'
However, if I try and set the value for the -d option curl cannot submit the request to the server.
var="{\"LED\":{ \"language\":\"CHINESE\"}, \"text\":\"1928年11月22日\"}"
# No Response
curl http://localhost:9020/xxx/xxxx/ -H Content-type:application/json
-H Accept:application/json -d $var
Can anyone explain why the latter does not seem to work? I have tried a variety of escape characters, but no luck with that either.

You need to double-quote the variable:
curl http://localhost:9020/xxx/xxxx/ -H Content-type:application/json \
-H Accept:application/json -d "$var"
Without the double-quoting, the value is split on spaces,
and the shell interprets that as multiple additional arguments to curl,
but you need that value to be a single argument.
That's what double-quoting will achieve.

Resources