I have a web service that expects some parameters. I want to pass one of these parameters (named "data") to curl via stdin.
I tried
echo -n "some data" | curl -d x="foo" -d y="bar" -d data=#- "http://somewhere"
which doesn't work, as the value of data then is "#-" instead of "some data".
Is it possible to use the # in curl to associate the input from stdin with a specific parameter?
edit: My goal is to chain multiples web services so the data I pass will be the output of another curl call.
Use -d #-
E.g.:
echo '{"text": "Hello **world**!"}' | curl -d #- https://api.github.com/markdown
Output:
<p>Hello <strong>world</strong>!</p>
Is this not possible?
curl -d x="foo" -d y="bar" -d data="#some data" "http://somewhere"
Or
curl -d x="foo" -d y="bar" -d data="#$(echo "some data")" "http://somewhere"
echo "some data" can be another command or file input: $(<somefile).
You could also save your data in a variable and then use it like this:
VAR="some data"; curl -d x="foo" -d y="bar" -d data=$VAR "http://somewhere"
Related
status=$(curl -i -s -H "Authorization: token abc123" https://www.some_url.com | awk 'NR==18')
if [ "$status" != 200 ]; then
echo "$status"
fi
The above code is working and echo is printing the output in my console.
But when I use discord webhook, it doesn't send message there and throws following error: {"code": 50109, "message": "The request body contains invalid JSON."}
webhook=https://discord.com/api/webhooks/abc123
status=$(curl -i -s -H "Authorization: token abc123" https://www.some_url.com | awk 'NR==18')
if [ "$status" != 200 ]; then
curl -H "Content-Type: application/json" -X POST -d '{"content":"'"$status"'"}' $webhook
fi
I can send a normal message through webhook but not this curl output. The output is suppose to be HTML error message.
Even if I change awk 'NR==18' to awk 'NR==1', it's not sending even the one line output which is suppose to be something like: HTTP/1.1 401 Unauthorized
I have a discord bot (webhook) running on a raspberry pi from a bash script, so I think it is the same thing you are looking for
sendcontent='{"username": "Botname", "content": "'"${messagearray[#]}"'"}'
curl -H "Content-Type: application/json" -d "$sendcontent" "$webhookurl"
I would suggest the following change to your code:
messagearray=($(curl -i -s -H "Authorization: token abc123" https://www.some_url.com | awk 'NR==18'))
EDIT:
It seems that there are nasty invisible characters in the curl+awk command return, therefore the above solution needs to be modified as below
(This bash file test with the provided url from comments)
#!/bin/bash
webhookurl="https://mywebhook"
messagearray=($(curl -i -s https://www.google.com/ | awk 'NR==1'))
#this next line removes nasty characters that give json errors (keeps only letters, numbers SPACE and -)
message=`echo ${messagearray[#]} | sed 's/[^0-9A-Za-z -]*//g'`;
sendcontent='{"username": "Botname", "content": "'"$message"'"}'
curl -H "Content-Type: application/json" -d "$sendcontent" "$webhookurl"
This successfully sends HTTP/2 200 in discord
the removal is an adaptation of:
How to remove a newline from a string in Bash
Hi there anyone there having the same trouble like mine?
whenever I cURL the $list from the list.txt it just displaying {} which is a blank response from the API does my code should be really working properly or it is just a bug?
I know the $list is working because I can update the database status
Please this is a bit urgennnnttt :(
#! /bin/bash
filename=/var/lib/postgresql/Script/list.txt
database='dbname'
refLink='URL'
authorization='Authorization: Basic zxc'
expireDate=$(date -d "+3 days")
body="Message."
while IFS=' ' read -r list
do
wow=$(curl --location --request POST $refLink \
--header 'Authorization: Basic $authorization' \
--header 'Content-Type: application/json' \
--data-raw '{
"title":"Expiration Notice",
"body":"$body",
"min" :[{"mobileNumber" : "$list"}],
"type" : "Notification",
"action_type" : "NotificationActivity"}')
echo "result: '$result'"
RESP=$(echo "$result" | grep -oP "^[^a-zA-Z0-9]")
echo "RESP:'$RESP'"
echo $body
#echo $wow >> logs.txt
psql -d $database -c "UPDATE tblname SET status='hehe' WHERE mobile='$list'"
done < $filename
Your "$list" JSON entry is not populated with the content of the $list variable because it is within single quotes of the --data-raw curl parameter.
What you need is compose your JSON data for the query before-hand, preferably with the help of jq or some other JSON processor, before sending it as argument to the curl's POST request.
Multiple faults in your scripts (not exhaustive):
Shebang is wrong with a space #! /bin/bash
expireDate=$(date -d "+3 days") return date in locale format and this may not be what you need for your request.
The request and the response data are not processed with JSON grammar aware tools. grep is not appropriate for JSON data.
Some clues but cannot fix your script more without knowing more about the API answers and functions you use.
Anyway here is how you can at least compose a proper JSON request:
#!/usr/bin/env bash
filename='/var/lib/postgresql/Script/list.txt'
database='dbname'
refLink='URL'
authorization='zxc'
expireDate=$(date -R -d "+3 days")
body="Message."
while IFS=' ' read -r list; do
raw_json="$(
jq -n --arg bdy "$body" --arg mobN "$list" \
'.action_type="NotificationActivity"|.title="Expiration Notice"|.type="Notification"|.body=$bdy|.min[0].mobileNumber=$mobN|.'
)"
json_reply="$(curl --location --request POST "$refLink" \
--header "Authorization: Basic $authorization" \
--header 'Content-Type: application/json' \
--data-raw "$raw_json")"
echo "json_reply: '$json_reply'"
echo "$body"
# psql -d "$database" -c "UPDATE tblname SET status='hehe' WHERE mobile='$list'"
done <"$filename"
This is the script i used to export some documents to elasticsearch but no luck
#!/bin/ksh
set -v
trap read debug
date=$(date +%Y-%m-%d);
echo $date;
config_file="/home/p.sshanm/reports_elastic.cfg";
echo $config_file;
URL="http://p-acqpes-app01.wirecard.sys:9200/reports-"$date"";
echo $URL;
find /transfers/documents/*/done/ -type f -name "ABC-Record*_${date}*.csv"|
while IFS='' read -r -d '' filename
do
echo "filename : ${filename}"
var=$(base64 "$filename"| perl -pe 's/\n//g');
#if i use below it will fail as argument too long , so i used with curl option #
# var1= $(curl -XPUT 'http://localhost:9200/reports-'$date'/document/reports?pipeline=attachment&pretty' -d' { "data" : "'$var'" }')
var1=$(curl -X PUT -H "Content-Type: application/json" -d #- "$URL" >>CURLDATA
{ "data": "$var" }
CURL_DATA)
done;
If i use below it as
var1= $(curl -XPUT 'http://localhost:9200/reports-'$date'/document/reports?pipeline=attachment&pretty' -d' { "data" : "'$var'" }')
will fail as below, so i used with curl option #
argument too long
Your syntax to read from stdin is wrong, the here-doc string should have been (<<) and the de-limiters are mis-matching use CURL_DATA at both places.
curl -X PUT -H "Content-Type: application/json" -d #- "$URL" <<CURL_DATA
{ "data": "$var" }
CURL_DATA
I am trying to build a simple server with netcat and curl.
netcat gets data coming from a port and then runs a curl command as follows to send the data to a webservice.
nc -l -k 2233 | while read x ; do curl -X POST -H "Content-Type: application/json" -d '{"DATA": `echo $x` }' https://example.com/FEP ; done
for some reason, the echo $x is not being evaluated to the read value.
You need to move $x out of the ' delimited string, e.g. -d '{"DATA":"'$x'" }'.
Special characters in single quoted strings are never honored.
See the example:
x=text
echo single: '$x'
echo double: "$x"
prints
single: $x
double: text
There is no need to echo $x; just use
curl [options] '{"DATA": '$x' }' https://example.com/FEP; done
So, i'm trying to write a script to create a server on Rackspace Cloud. I've been able to get the script to successfully create a server. Although, after the server is created i need to get a couple of pieces of data out of the output and store them in a variable for later use in the same script (going to do a couple things on the new server after creation).
Here is the script i'm using to create the server:
#!/bin/bash
# Ask user to continue or not
read -p "You are about to setup a new Rackspace cloud server. Would you like to continue (y/n)? " cont
if [ "$cont" == "y" ]; then
echo "Starting server setup script..."
# Ask questions to get login creds
read -p "Rackspace Username? " username
read -p "Rackspace API Key? " apikey
# Get Rackspace token
echo "Getting token..."
token=$(curl -s -X POST https://auth.api.rackspacecloud.com/v2.0/tokens \
-d '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"'"$username"'", "apiKey":"'"$apikey"'" } } }' \
-H "Content-type: application/json" \
| python -m json.tool \
| python -c 'import sys, json; print json.load(sys.stdin)["access"]["token"]["id"]')
echo "...done!"
# Get Rackspace account id
echo "Getting account id..."
account=$(curl -s -X POST https://auth.api.rackspacecloud.com/v2.0/tokens \
-d '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"'"$username"'", "apiKey":"'"$apikey"'" } } }' \
-H "Content-type: application/json" \
| python -m json.tool \
| python -c 'import sys, json; print json.load(sys.stdin)["access"]["token"]["tenant"]["id"]')
echo "...done!"
# Create a new Rackspace cloud server
echo "Creating a new Rackspace cloud server..."
serverpassword=$(curl -s -X POST https://dfw.servers.api.rackspacecloud.com/v2/$account/servers \
-H "Content-Type: application/json" \
-H "X-Auth-Token: $token" \
-H "X-Auth-Project-Id: test-project" \
-T server_build.json \
| python -m json.tool \
| python -c 'import sys, json; print json.load(sys.stdin)["server"]["adminPass"]')
echo "...done!"
echo "Your new server has been created!"
read -p "Would you like to continue on to setting up your new server (y/n)? " cont2
if [ "$cont2" == "y" ]; then
exit
else
echo "Here is your root password for this server (you must write this down now): $serverpassword"
exit
fi
else
echo "You have chosen not to setup a new Rackspace server."
exit
fi
You can see that the token and account variables are being set and used in the creation curl command. The last curl command outputs something like this: (removed real values)
{
"server": {
"OS-DCF:diskConfig": "AUTO",
"adminPass": "************",
"id": "************",
"links": [
{
"href": "https://dfw.servers.api.rackspacecloud.com/v2/...",
"rel": "self"
},
{
"href": "https://dfw.servers.api.rackspacecloud.com/...",
"rel": "bookmark"
}
]
}
}
I'd like to be able to store both the adminPass and id in two different variables. How can i do this? I also know that by solving this issue i will be able to refactor the script when getting token and account values by using one curl command instead of two different ones.
Thanks for any help!
I appreciate all the help. I was able to figure out what i needed to do. Maybe my original question wasn't worded in a way to get the best answers but hopefully this resolution will explain better what i was trying to accomplish.
I ended up using these commands for the 3rd curl command and to set the variables i needed:
serverinfo=$(curl -s https://dfw.servers.api.rackspacecloud.com/v2/731174/servers \
-X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: $token" \
-H "X-Auth-Project-Id: test-project" \
-T server_build.json | python -m json.tool)
serverpass=$(echo $serverinfo | python -c 'import sys, json; print json.load(sys.stdin)["server"]["adminPass"]')
This allowed me to run the curl command to setup the server and then grab the adminPass from the output of that command and set it to the serverpass variable for later use.
Using this concept i was also able to remove the 2nd curl command since it was a duplicate command just to set a variable. I can now use these commands to rub the curl once and set both variables from its output:
tokensinfo=$(curl -s -X POST https://auth.api.rackspacecloud.com/v2.0/tokens \
-d '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"'"$username"'", "apiKey":"'"$apikey"'" } } }' \
-H "Content-type: application/json")
token=$(echo $tokensinfo | python -c 'import sys, json; print json.load(sys.stdin)["access"]["token"]["id"]')
account=$(echo $tokensinfo | python -c 'import sys, json; print json.load(sys.stdin)["access"]["token"]["tenant"]["id"]')
Hope that better explains what i was going for. Thanks for the help, it did help me arrive at this solution.
you can grab adminpass and id from the output and store in new variable like
var1=$(grep "adminPass" output.txt | cut -d: --complement -f1)
var2=$(grep "id" output.txt | cut -d: --complement -f1)
echo -e $var1
echo -e $var2
for here i assume the third command output you redirected to output.txt file.