For now I'm using this in my script:
set -euxo pipefail
This will make my bash script fail immediatly if there is an error in my pipe.
When I leave this option my script will run fully and end with exit 0 (no error).
I want to have a mix of both. I want to end the whole script but have exit 1; at the end if there was an error in a pipe.
My script looks like this:
#!/bin/bash
set -euxo pipefail
curl --fail --compressed -u "$CREDS" -X GET --header 'Accept: xxx' --header 'Accept-Language: de-DE' 'https://api/call1' | jq -S "." > "output1.json"
curl --fail --compressed -u "$CREDS" -X GET --header 'Accept: xxx' --header 'Accept-Language: de-DE' 'https://api/call2' | jq -S "." > "output2.json"
cat output1.json
cat output2.json
So I don't want to exit the script if call1 fails. If call1 fails I want to go to call2 and the cat commands before exiting the script with exit code 1.
Don't use set -e as that will make script exit upon first error. Just save your exit codes after call1 and call2 and exit with appropriate exit code after cat commands:
#!/usr/bin/env bash -ux
set -uxo pipefail
curl --fail --compressed -u "$CREDS" -X GET --header 'Accept: xxx' --header 'Accept-Language: de-DE' 'https://api/call1' | jq -S "." > "output1.json"
ret1=$?
curl --fail --compressed -u "$CREDS" -X GET --header 'Accept: xxx' --header 'Accept-Language: de-DE' 'https://api/call2' | jq -S "." > "output2.json"
ret2=$?
cat output1.json
cat output2.json
exit $((ret1 | ret2))
Subshells.
set -euo pipefail
export SHELLOPTS
(
set -euo pipefail
curl --fail --compressed -u "$CREDS" -X GET --header 'Accept: xxx' --header 'Accept-Language: de-DE' 'https://api/call1' | jq -S "." > "output1.json"
) && res1=$? || res1=$?
(
set -euo pipefail
curl --fail --compressed -u "$CREDS" -X GET --header 'Accept: xxx' --header 'Accept-Language: de-DE' 'https://api/call2' | jq -S "." > "output2.json"
) && res2=$? || res2=$?
if (( res1 != 0 || res2 != 0 )); then
echo "Och! Command 1 failed or command 2 failed, what is the meaning of life?"
exit 1;
fi
Subshell let's you grab the return value of the commands executed in it.
Related
I am trying to check the output from an http call, if the variable matches the a string I want to terminate the loop. Unfortunately, that is not happening in my case it is not terminating the loop. Here is my code :
while [[ "$BUILD_STATUS" != "SUCCESS" ]];
do
http_response_1=$(curl -s -o response_1.json -w "%{http_code}" ${CREATE_BUILD_URL}/${BUILD_CODE}/progress -H 'Authorization: Token' -H 'Content-Type:application/json')
BUILD_PERCENTAGE=$(jq -r .percentage response_1.json)
BUILD_STATUS=$(jq .buildStatus response_1.json)
echo "$BUILD_STATUS"
rm response_1.json
echo "File Removed"
sleep 10
done
Directly have jq make the string comparison and return a status accordingly:
#!/usr/bin/env sh
while
build_percentage="$(
curl \
--silent \
--write-out "%{http_code}" \
--header 'Authorization: Token' \
--header 'Content-Type:application/json' \
--url "${CREATE_BUILD_URL}/${BUILD_CODE}/progress" |
jq \
--exit-status \
--raw-output \
'.percentage,
if .buildStatus != "SUCCESS"
then empty
else halt_error(1)
end
' 2>/dev/null
)"
do
printf 'Build percentage: %d%%\n' "$build_percentage"
sleep 10
done
printf 'Build percentage: %d%%\nStatus: SUCCESS!\n' "$build_percentage"
Goal:
I'm using a bash CURL script to connect to the Cloudflare APIv4. The goal is to update an A-record. My script:
# Get current public IP
current_ip=curl --silent ipecho.net/plain; echo
# Update A record
curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONEIDHERE/dns_records/DNSRECORDHERE" \
-H "X-Auth-Email: EMAILHERE" \
-H "X-Auth-Key: AUTHKEYHERE" \
-H "Content-Type: application/json" \
--data '{"id":"ZONEIDHERE","type":"A","name":"example.com","content":"'"${current_ip}"'","zone_name":"example.com"}'
Problem:
The current_ip variable is not printed when I call it in my script. The output will be "content" : "" and not "content" : "1.2.3.4".
I used other stackoverflow posts and I'm trying to follow their examples but I think I'm still doing something wrong, just can't figure out what. :(
Using jq for this, as Charles Duffy's answer suggests, is a very good idea. However, if you can't or do not want to install jq here is what you can do with plain POSIX shell.
#!/bin/sh
set -e
current_ip="$(curl --silent --show-error --fail ipecho.net/plain)"
echo "IP: $current_ip"
# Update A record
curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONEIDHERE/dns_records/DNSRECORDHERE" \
-H "X-Auth-Email: EMAILHERE" \
-H "X-Auth-Key: AUTHKEYHERE" \
-H "Content-Type: application/json" \
--data #- <<END;
{
"id": "ZONEIDHERE",
"type": "A",
"name": "example.com",
"content": "$current_ip",
"zone_name": "example.com"
}
END
The reliable way to edit JSON from shell scripts is to use jq:
# set shell variables with your contents
email="yourEmail"
authKey="yourAuthKey"
zoneid="yourZoneId"
dnsrecord="yourDnsRecord"
# make sure we show errors; --silent without --show-error can mask problems.
current_ip=$(curl --fail -sS ipecho.net/plain) || exit
# optional: template w/ JSON content that won't change
json_template='{"type": "A", "name": "example.com"}'
# build JSON with content that *can* change with jq
json_data=$(jq --arg zoneid "$zoneid" \
--arg current_ip "$current_ip" \
'.id=$zoneid | .content=$current_ip' \
<<<"$json_template")
# ...and submit
curl -X PUT "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records/$dnsrecord" \
-H "X-Auth-Email: $email" \
-H "X-Auth-Key: $authKey" \
-H "Content-Type: application/json" \
--data "$json_data"
I'm using the following script to purge cache from cdn,
#!/bin/bash
## API keys ##
zone_id=""
api_key=""
login_id=""
akamai_crd=""
## URL ##
urls="$1"
[ "$urls" == "" ] && { echo "Usage: $0 url"; exit 1; }
echo "Purging $urls..."
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/${zone_id}/purge_cache" \
-H "X-Auth-Email: ${login_id}" \
-H "X-Auth-Key: ${api_key}" \
-H "Content-Type: application/json" \
--data "{\"files\":[\"${urls}\"]}"
#echo "CF is done now purging from Akamai ..."
echo "..."
curl -v -s https://api.ccu.akamai.com/ccu/v2/queues/default -H "Content-Type:application/json" -d '{"objects":["$urls"]}' -u $akamai_crd
The 1st part for cloudflare is working fine the 2nd part when I pass it to Akamai
["$urls"]
I keep getting an error and it's passing the url as an argument it returns the variable itself ($urls) not the arg value.
I ran the script as following:
sh +x script.sh url
Any advise here?
First i would change this:
SCRIPTNAME=$(basename "$0")
...
if [ $# != 1 ] then
echo "Usage: $SCRIPTNAME url"
exit
fi
$urls="$1"
Change your second curl command as the following (you need to escape the quotes):
--data "{\"files\":[\"${urls}\"]}"
Avoid creating JSON by hand like this; you can't guarantee that the resulting JSON is properly escaped. Use a tool like jq instead.
#!/bin/bash
## API keys ##
zone_id=""
api_key=""
login_id=""
akamai_crd=""
## URL ##
url=${1:?Usage: $0 url}
headers=(
-H "X-Auth-Email: $login_id"
-H "X-Auth-Key: $api_key"
-H "Content-Type: application/json"
)
purge_endpoint="https://api.cloudflare.com/client/v4/zones/${zone_id}/purge_cache"
echo "Purging $url..."
jq -n --arg url "$url" '{files: [$url]}' |
curl -X DELETE "$purge_endpoing" "${headers[#]}" --data #-
#echo "CF is done now purging from Akamai ..."
echo "..."
jq -n --arg url "$url" '{objects: [$url]}' |
curl -v -s -H "Content-Type:application/json" -d #- -u "$akamai_crd"
If I run the following command, everything works great:
curl -s -o /dev/null -w %{http_code} \
http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
When I put this command into a variable in a /bin/sh script, and execute it, I get "HTTP 415 Unsupported Media Type" showing up in the server logs:
#!/bin/sh
read -d '' CMD << EOF
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
EOF
`$CMD`
I verified the command by printing it out and it looks example like the manual one I run:
#!/bin/sh
read -d '' CMD << EOF
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
EOF
echo $CMD
Output:
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com --header "Content-type: application/json" --request PUT --data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
For me it worked this way:
#!/bin/sh
read -d '' CMD << EOF
`/usr/bin/curl -s http://localhost:8080/someurl \
--header "Content-type: application/json"`
EOF
echo $CMD
What if evaluate it inside of EOF?
I'm trying to get both the HTTP code and the output of a curl command as part of a shell script, but in turn I'm trying to set both of them as variables. My code (which uses the pushbullet API) looks like this:
CURL_OUTPUT="$(exec 3>&1; \
HTTP_CODE="$(curl -s -S \
-w "%{http_code}" -o >(cat >&3) \
--header 'Authorization: Bearer '"$ACCT_TOKEN" \
-X POST https://api.pushbullet.com/v2/pushes \
--header 'Content-Type: application/json' \
--data-binary "$JSON")" \
)"
which should theoretically set $CURL_OUTPUT to the JSON returned by curl and $HTTP_CODE to the status code I get. Instead, I get only $CURL_OUTPUT; $HTTP_CODE is empty.
However, if I don't do the outer nest, like so:
exec 3>&1; \
HTTP_CODE="$(curl -s -S \
-w "%{http_code}" -o >(cat >&3) \
--header 'Authorization: Bearer '"$ACCT_TOKEN" \
-X POST https://api.pushbullet.com/v2/pushes \
--header 'Content-Type: application/json' \
--data-binary "$JSON")" \
The command works as expected; I get the JSON redirected to stdout, and the status code ends up in $HTTP_CODE.
So is it possible to get both outputs? Can I assign variables within a command substitution line?