Curl command with option -d issue - bash

Trying to send some data via API call (curl) but having issue with the format of the curl command:
serviceMsg="$(cat /tmp/response_time)"
perfData=$(/bin/echo "${serviceMsg}" | /bin/sed 's/,//g')
StatusCode=0
curl -k -s -v -u user:password -H 'Accept: application/json' -X POST 'https://localhost:5665/v1/actions/process-check-result?service=ExternalURL!ResponseTime' -d "{ \"exit_status\": $StatusCode, \"plugin_output\": \"${perfData}\", \"performance_data\": [ ${perfData} ]}"
Here is the error I am getting:
* Connection #0 to host localhost left intact
{
"error": 400.0,
"status": "Invalid request body: Error: lexical error: invalid character inside string.\n { \"exit_status\": 0, \"plugin_ou\n (right here) ------^\n\n"
}
what I am missing ?

Don't hand-code JSON: use jq to generate it for you:
perfData=$(sed 's/,//g' /tmp/response_time)
json=$(jq -n --arg sc "$StatusCode" --arg pd "$perfData" \
'{exit_status: $sc, plugin_output: $pd, performance_data: [ $pd ]}')
url='https://localhost:5665/v1/actions/process-check-result?service=ExternalURL!ResponseTime'
curl -ksv -u user:password -H 'Accept: application/json' -X POST -d "$json" "$url"

Related

curl: (3) URL using bad/illegal format or missing URL AND HTTP/1.1 403 Forbidden

I have a problem with this code. It states that there is a URL issue but I don't see that as right. I am not responsible for the server though. I just wanted to test some IoT properties with basic linux commands.
#!/bin/bash
a=Client secret
c=Custom domain
b=Client id:
#!/bin/bash
response=$(curl -X POST --user $b:$a "https://${c}.auth.eu-central-1.amazoncognito.com/oauth2/token?grant_type=client_credentials" -H 'Content-Type: application/x-www-form-urlencoded')
token=$(jq -r '.access_token' <<< "$response")
secondsValid=$(jq -r '.expires_in' <<< "$response")
refreshToken=$(jq -r '.refresh_token' <<< "$response")
for i in {1..4}
do
declare -i TIME
TIME=$(date +%s)
TEMP=$(sensors -j | jq '."cpu_thermal-virtual-0"."temp1"."temp1_input"')
curl -i -k -X POST -H "Authorization: $token"-H "Content-Type: application/json" --data '{ "id":"2","timestamp":"'${TIME}'","data":"'${TEMP}'"}' https://dv7knsjzph.execute-api.eu-central-1.amazonaws.com/prod/boxtronic-devices/2/data/
sleep 1
echo "$refreshToken"
echo "$secondsValid"
echo "$TIME"
done
Anyone knows why I am getting the error ?

Parse curl response to variable and use it in curl

I have the following script:
#!/bin/bash
TOKEN=$(curl -isX POST 'http://localhost:3005/auth/tokens' \
-H 'Content-Type: application/json' \
-d '{
"name": "test#test.de",
"password": "1234"
}' | grep X-Subject-Token | sed "s/X-Subject-Token: //g")
echo $TOKEN
curl --trace test.txt -X POST "http://localhost:3005/v1/users" \
-H "Content-Type: application/json" \
-H "X-Auth-token: $TOKEN" \
-d '{
"user": {
"username": "alice",
"email": "alice#test.com",
"password": "test"
}
}'
The command echo $TOKEN is printing the right result (something like 35be3d05-7f80-4b11-ad20-7a7110e9d3a7). From the last curl request I get the following error from curl:
curl: (52) Empty reply from server
If I write above the last curl request TOKEN="35be3d05-7f80-4b11-ad20-7a7110e9d3a7" the request is working. So I guess there is something wrong with the TOKEN variable.
Kindly Regards
EDIT:
Output from declare -p TOKEN :
"eclare -- TOKEN="6770806a-1230-4f64-b519-1841e9deb5f1
I had to remove an carriage return. Thanks to #chepner!
Solution:
TOKEN=$(curl -isX POST 'http://localhost:3005/auth/tokens' \
-H 'Content-Type: application/json' \
-d '{
"name": "test#test.de",
"password": "1234"
}' | grep X-Subject-Token | sed "s/X-Subject-Token: //g" | tr -d '\r')

kraken private API BASH, curl and openssl

I am using bash and curl to operate the Kraken API.
When I try to AddOrder, I get the response :
{"error":["EGeneral:Invalid arguments"]}
When I use the same code to find the Balance, I don't get an error.
The URL I am using looks like so :
https://api.kraken.com/0/private/AddOrder?pair=XXBTZUSD&type=buy&ordertype=limit&price=6000&volume=1&userref=1537588642&validate=1
The code looks like so (with the final openssl and curl command at the very end) :
FUNC="AddOrder"
FN="/tmp/kraken$FUNC.json"
# URL
PRICE=$1
VOL=$2
ORDERTYPE=limit
ID=`date +%s`
URLPATH="/0/private/$FUNC"
URLPATH="$URLPATH?pair=XXBTZUSD&type=buy&ordertype=$ORDERTYPE&price=$PRICE&volume=$VOL&userref=$ID&validate=1"
echo $URLPATH
curlPost $URLPATH $FUNC $FN
curlPost(){
# API Key:
APIKEY=`cat creds.key`
# Private Key:
PRIVATEKEY=`cat creds.private`
URLHOST="https://api.kraken.com"
NONCE=$(date +%s%N)
echo ${1}
echo -n "${1}" > /tmp/kraken.$2.bin
echo -n "${NONCE}nonce=${NONCE}" | openssl sha256 -binary >> /tmp/kraken.$2.bin
PRIVATEKEYHEX=$(echo -n "${PRIVATEKEY}" | base64 -d | hexdump -v -e '/1 "%02X"');
cat /tmp/kraken.$2.bin | openssl sha512 -binary -mac HMAC -macopt hexkey:${PRIVATEKEYHEX} > sign.$2.bin
APISIGN=$(cat sign.$2.bin | base64 -w 0)
curl -X POST -H "Accept: application/json" -H "API-Key: ${APIKEY}" -H "API-Sign: ${APISIGN}" -d "nonce=$NONCE" ${URLHOST}${URLPATH} > $3
}
I have worked out the solution. The curl argumnet -d for data, like so :
URLPATH="/0/private/$FUNC"
DATA="pair=XXBTZUSD&type=buy&ordertype=$ORDERTYPE&price=$PRICE&volume=$VOL&userref=$ID&validate=1"
if [ -z "$DATA" ]; then
curl -X POST -H "Accept: application/json" -H "API-Key: ${APIKEY}" -H "API-Sign: ${APISIGN}" -d "nonce=$NONCE" "${URLHOST}${URLPATH}"
else
curl -d ${DATA} -X POST -H "Accept: application/json" -H "API-Key: ${APIKEY}" -H "API-Sign: ${APISIGN}" -d "nonce=$NONCE" "${URLHOST}${URLPATH}"
fi

How can I post a json string with curl that has characters that need escaping?

I've got a shell script I've been using to post stuff to a hipchat channel. It works ok until I try and send a message that has characters that need escaping. I run the command like so (note the extra backslash in there to cause a problem)
/usr/local/bin/hipchatmsg.sh "my great message here \ " red
And my code in my bash script (hipchatmsg.sh) that matters is this:
# Make sure message is passed
if [ -z ${1+x} ]; then
echo "Provide a message to create the new notification"
exit 1
else
MESSAGE=$1
fi
// send locally via curl
/usr/bin/curl -H "Content-Type: application/json" \
-X POST \
-k \
-d "{\"color\": \"$COLOR\", \"message_format\": \"text\", \"message\": \"$MESSAGE\" }" \
$SERVER/v2/room/$ROOM_ID/notification?auth_token=$AUTH_TOKEN &
// $server and $room are defined earlier
exit 0
If I try and run the command above with any characters that need escaping, I will get an error like this:
{
"error": {
"code": 400,
"message": "The request body cannot be parsed as valid JSON: Invalid \\X escape sequence u'\\\\': line 1 column 125 (char 124)",
"type": "Bad Request"
}
}
I found something kind of similar on here where the best advice was to try sending the curl post with --data-urlencode, so I tried like this:
/usr/bin/curl -H "Content-Type: application/json" \
-X POST \
-k \
-d --data-urlencode "{\"color\": \"$COLOR\", \"message_format\": \"text\", \"message\": \"$MESSAGE\" }" \
$SERVER/v2/room/$ROOM_ID/notification?auth_token=$AUTH_TOKEN &
But this had no effect.
What am I missing here?
The easiest thing to do is use a program like jq to generate the JSON; it will take care of escaping what needs to be escaped.
jq -n --arg color "$COLOR" \
--arg message "$MESSAGE" \
'{color: $color, message_format: "text", message: $message}' |
/usr/bin/curl -H "Content-Type: application/json" \
-X POST \
-k \
-d#- \
$SERVER/v2/room/$ROOM_ID/notification?auth_token=$AUTH_TOKEN &
The argument #- to -d tells curl to read from standard input, which is supplied from jq via the pipe. The --arg options to jq make available JSON-encoded strings to the filter, which is simply a JSON object expression.

How to use a variable in a CURL request with bash?

Goal:
I'm using a bash CURL script to connect to the Cloudflare APIv4. The goal is to update an A-record. My script:
# Get current public IP
current_ip=curl --silent ipecho.net/plain; echo
# Update A record
curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONEIDHERE/dns_records/DNSRECORDHERE" \
-H "X-Auth-Email: EMAILHERE" \
-H "X-Auth-Key: AUTHKEYHERE" \
-H "Content-Type: application/json" \
--data '{"id":"ZONEIDHERE","type":"A","name":"example.com","content":"'"${current_ip}"'","zone_name":"example.com"}'
Problem:
The current_ip variable is not printed when I call it in my script. The output will be "content" : "" and not "content" : "1.2.3.4".
I used other stackoverflow posts and I'm trying to follow their examples but I think I'm still doing something wrong, just can't figure out what. :(
Using jq for this, as Charles Duffy's answer suggests, is a very good idea. However, if you can't or do not want to install jq here is what you can do with plain POSIX shell.
#!/bin/sh
set -e
current_ip="$(curl --silent --show-error --fail ipecho.net/plain)"
echo "IP: $current_ip"
# Update A record
curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONEIDHERE/dns_records/DNSRECORDHERE" \
-H "X-Auth-Email: EMAILHERE" \
-H "X-Auth-Key: AUTHKEYHERE" \
-H "Content-Type: application/json" \
--data #- <<END;
{
"id": "ZONEIDHERE",
"type": "A",
"name": "example.com",
"content": "$current_ip",
"zone_name": "example.com"
}
END
The reliable way to edit JSON from shell scripts is to use jq:
# set shell variables with your contents
email="yourEmail"
authKey="yourAuthKey"
zoneid="yourZoneId"
dnsrecord="yourDnsRecord"
# make sure we show errors; --silent without --show-error can mask problems.
current_ip=$(curl --fail -sS ipecho.net/plain) || exit
# optional: template w/ JSON content that won't change
json_template='{"type": "A", "name": "example.com"}'
# build JSON with content that *can* change with jq
json_data=$(jq --arg zoneid "$zoneid" \
--arg current_ip "$current_ip" \
'.id=$zoneid | .content=$current_ip' \
<<<"$json_template")
# ...and submit
curl -X PUT "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records/$dnsrecord" \
-H "X-Auth-Email: $email" \
-H "X-Auth-Key: $authKey" \
-H "Content-Type: application/json" \
--data "$json_data"

Resources