How to reuse a variable in shell script when used in curl? - bash

Depending on certain conditions I want to use JWT else I want to provide path to certs. Thus in my shell script this is the code:
if /* some condition */
authorization='-H "'Authorization': 'Bearer' ${JWT}"'
else
authorization="--cert "${ADMIN_CERT_PATH}" --key "${ADMIN_KEY_PATH}""
Now the curl request should be:
curl -H "Authorization: Bearer 348129" for if condition
curl --cert /Users/.../admin_cert --key /Users/../admin_key .. for else path
In order to get that output I need to use the following format in my shell script for if condition
response_code="$(curl -s -o /dev/null -w "%{http_code}" "$authorization" "$status_url")"
and following format for else code:
response_code="$(curl -s -o /dev/null -w "%{http_code}" $authorization "$status_url")"
Note:
I need $authorization variable quoted in first case and unquoted in the else case.
I do not want to write 2 different curl commands instead reuse the authorization variable.
Thus, i need to modify the way I have declared my authorization variable such that I can write any one of the curl commands only once which works for both if and else cases.

curl supports a way to pass command line parameters in a file that I have used before when I have complex parameters. The idea is to place the complex command-line parameters into a simple text file and instruct curl to read parameters from it using --config parameter.
In this case the shell script would look something like the following.
#!/bin/sh
## "safely" create a temporary configuration file
curlctl=$(mktemp -q -t $(basename "$0"))
if test $? -ne 0
then
echo "$0: failed to create temporary file, exiting."
exit 75 # EX_TEMPFAIL
fi
trap 'rm "$curlctl"' 0
## write parameters used in all cases
cat>>"$curlctl" <<EOF
output = /dev/null
silent
write-out = %{http_code}
EOF
## append conditional parameters
if test "$some" = 'condition'
then
printf 'header = "Authorization: Bearer %s"\n' "$JWT" >> "$curlctl"
else
echo "cert = $ADMIN_CERT_PATH" >> "$curlctl"
echo "key = $ADMIN_KEY_PATH" >> "$curlctl"
fi
# uncomment to see what the config file looks like
# cat "$curlctl" | sed 's/^/curl config: /'
response_code=$(curl --config "$curlctl" http://httpbin.org/get)
echo "response code: $response_code"
The first few lines set up a temporary file that is deleted when the shell script exits. If you are already using trap then your cleanup will probably be more complex.

When you are using a shell that supports arrays, you can avoid the need for a temporary configuration file.
curl_opts=(-s -o /dev/null -w "%{http_code}")
if /* some condition */
curl_opts+=(-H "Authorization: Bearer $JWT")
else
curl_opts+=(--cert "$ADMIN_CERT_PATH" --key "$ADMIN_KEY_PATH")
fi
...
response_code="$(curl "${curl_opts[#]}" "$status_url")"

Related

bash ldapsearch take input from file [duplicate]

This question already has answers here:
How to apply shell command to each line of a command output?
(9 answers)
Closed 5 months ago.
I have a simple ldapsearch bash script to return the user email when searched by ID. I made it take and argument as its input since at the time I only needed to run it once or twice.
I'm wondering can I adapt it and take input from a file like .txt and append the outputs to another file.
This is what i have:
#!/bin/bash
if [ "$1" = "" ]; then
echo "how to: searchID.sh <userID>"
exit 1
fi
ldapsearch -x -b '' -LLL -h ldaphost.com -p 255 uid=$1 mail >> outputs.txt
Instead of running it manually like:
./searchID.sh I0FT45
I want it to take input from a file with many ID's like:
I0001F
IGLFK7
I37462
I4593N
And run it for all those entries.
Any help is very much appreciated
if your usernames are xargs "safe" (no space, no quote) then you can do something like this:
xargs -I {} \
ldapsearch -x -b '' -LLL -h ldaphost.com -p 255 uid={} mail \
< file.in \
>> file.out

Building a curl command with parameters that have to be URL-encoded

I want to send data to a server via curl. The script has to first build the curl command by appending several parameters. Something like this:
# set other params...
param4_name="param4"
param4_value="Hello world"
params=""
# append other params to $params...
if [[ ! -z "$param4_value" ]]; then
params="$params --data-urlencode \"$param4_name=$param4_value\" "
fi
response="$(curl \
$params \
--get \
$my_url/test)"
When I use this code, however, the server receives the curl request as:
HTTP-Request: GET /test?"param4=Hello HTTP/1.1
(The " is included in the key and " world" is missing in the value.)
If I adjust the code to use:
if [[ ! -z "$param4_value" ]]; then
params="$params --data-urlencode $param4_name="$param4_value" "
fi
The server receives this:
HTTP-Request: GET /test?param4=Hello HTTP/1.1
(" world" is missing in the value.)
How do I adjust the bash code so that the server receives this request instead:
HTTP-Request: GET /test?param4=Hello%20World HTTP/1.1
I'm amenable to using this solution. But is there a "neater" way to do this without resorting to using a function, an embedded curl call, or perl/sed/awk/etc?

Unable to pass variable in gremlin query in shell script

I am trying to connection to Neptune DB and getting vertices details using CURL command. I have shell script for it. But somehow variable data is not going through it gremlin query. I have one Orgid.txt file where tenantid is present and my shell script reading the file and passing it to "name" variable
#!/bin/bash
i=1
rm VerticesCount1
while IFS= read -r line
do
name="$line"
#echo "Orgid name is "$i": $name"
curl -X POST https://<Neptune_endpoint>:<port>/gremlin -d '{"gremlin":"g.V().has(\"system.tenantId\",\"$name\").count()"}' >> VerticesCount1
#printf "\n"
echo >> VerticesCount1
((i=i+1))
done < Orgid.txt
As with your other question I tested with a simple data file and it works fine. However, note how I changed the type of quotes used by curl.
i=1
while IFS= read -r line
do
name="$line"
curl -X POST https://mydbcluster.cluster-xxxxxxxxxxxx.us-east-1.neptune.amazonaws.com:8182/gremlin -d \
"{\"gremlin\":\"g.V().has('code','$name').count()\"}"
((i=i+1))
done < values.txt
which produces
{"requestId":"4e3e80ed-efcb-40a7-b92b-366c6f391d4e","status":{"message":"","code":200,"attributes":{"#type":"g:Map","#value":[]}},"result":{"data":{"#type":"g:List","#value":[{"#type":"g:Int64","#value":1}]},"meta":{"#type":"g:Map","#value":[]}}}{"requestId":"6a269b5b-32f6-49d2-a31d-c51dd52eba29","status":{"message":"","code":200,"attributes":{"#type":"g:Map","#value":[]}},"result":{"data":{"#type":"g:List","#value":[{"#type":"g:Int64","#value":1}]},"meta":{"#type":"g:Map","#value":[]}}}
it is working fine with this code.
while IFS= read -r line
do
name="$line"
#echo "Orgid name is "$i": $name"
curl -X POST https://<Neptune_endpoint>:<port>/gremlin -d '{"gremlin":"g.V().has(\"system.tenantId\",\"'$name'\").count()"}' >> VerticesCount1
echo >> VerticesCount1
done < Orgid.txt

Bash script to query VirusTotal

I have a hash file containing several md5 hashes.
I want to create a bash script to curl virustotal to check if the hashes are known.
#!/bin/bash
for line in "hash.txt";
do
echo $line; curl -s -X GET --url 'https://www.virustotal.com/vtapi/v2/file/report?apikey=a54237df7c5c38d58d2240xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxcc0a0d7&resource='$line'';
done
but not working.
Could you help me please?
Better use a while loop. Your for loop would only run once, because bash interpret it as a value, not a file. Try this:
while read -r line; do
echo "$line"
curl -s -X GET --url "https://www.virustotal.com/vtapi/v2/file/report?apikey=a54237df7c5c38d58d2240xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxcc0a0d7&resource=$line"
done <"/path/to/hash.txt"

Bash: Looping urls using curl, giving only first url output

What I want to do: I want to find all the products(URLs) which are not redirected.
To get the final URL after redirection I'm using curl command as follows:
curl -Ls -o /dev/null -w %{url_effective} "$URL"
This is working fine. Now I want to iterate over URLs to get which are the URLs that are not redirected and display them as output of program. I've the following code:
result=""
productIds=$1
for productId in $(echo $productIds | sed "s/,/ /g")
do
echo "Checking product: $productId";
URL="http://localhost/?go=$productId";
updatedURL=`curl -Ls -o /dev/null -w %{url_effective} "$URL"`
echo "URL : $URL, UpdatedUrl: $updatedURL";
if [ "$URL" == "$updatedURL" ]
then
result="$result$productId,";
fi
done
The curl command works only for the first product. But from 2nd to last product, all the URL and updatedURL are same. I can't understand the reason why? The productId is changing in every iteration, so I think it cannot be something related to caching.
Also, I've tried the following variant of curl also:
updatedURL=$(curl -Ls -o /dev/null -w %{url_effective} "$URL")
updatedURL="$(curl -Ls -o /dev/null -w %{url_effective} "$URL")"
Edit: After trying with debug mode and lot of different ways. I noticed a pattern i.e. If I manually hit the following on terminal:
curl -Ls -o /dev/null -w %{url_effective} "http://localhost/?go=32123"
Then in shell script these urls will work fine. But if I don't hit manually then curl will also not work for those products via shell script.
Just add #!/bin/bash to be the first line of shell. It does produce required output. Now invocation should be like this bash file.sh 123,456,789,221
Invocation via Bourne shell sh file.sh 123,456,789,221 does requires code changes.Do let me know if you require that too :)
I would suggest changing your loop to something like this:
IFS=, read -ra productIds <<<"$1"
for productId in "${productIds[#]}"; do
url=http://localhost/?go=$productId
num_redirects=$(curl -Ls -o /dev/null -w %{num_redirects} "$url")
if [ "$num_redirects" -eq 0 ]; then
printf 'productId %s has no redirects\n' "$productId"
fi
done
This splits the first argument passed to the script into an array, using a comma as the delimiter. The number of redirects is stored to a variable. When the number is zero, the message is printed.
I have to admit that I can't see anything inherently broken with your original approach so it's possible that there is something extra going on that we're not aware of. If you could provide a reproducible test case then we would be able to help you more effectively.

Resources