I need to write a bash script that takes CSV file and iterates row by row, sending each row to http://localhost:9999/myListener.
In other words the script should execute this code for each N-th row of CSV file:
curl -H "Content-Type: application/json" -X POST -d '{"col1":1,"col2":3,"col3":"value"}' http://localhost:9999/myListener
you could translate csv to json line by line with awk:
cat foo.csv | \
awk -F',' '{printf("{\"col1\": %s, \"col2\": %s, \"col3\": \"%s\"}\n", $1, $2, $3)}' | \
while read s
do
curl -H "Content-Type: application/json" -X POST -d "$s" http://localhost:9999/myListener
done
Related
Hi there anyone there having the same trouble like mine?
whenever I cURL the $list from the list.txt it just displaying {} which is a blank response from the API does my code should be really working properly or it is just a bug?
I know the $list is working because I can update the database status
Please this is a bit urgennnnttt :(
#! /bin/bash
filename=/var/lib/postgresql/Script/list.txt
database='dbname'
refLink='URL'
authorization='Authorization: Basic zxc'
expireDate=$(date -d "+3 days")
body="Message."
while IFS=' ' read -r list
do
wow=$(curl --location --request POST $refLink \
--header 'Authorization: Basic $authorization' \
--header 'Content-Type: application/json' \
--data-raw '{
"title":"Expiration Notice",
"body":"$body",
"min" :[{"mobileNumber" : "$list"}],
"type" : "Notification",
"action_type" : "NotificationActivity"}')
echo "result: '$result'"
RESP=$(echo "$result" | grep -oP "^[^a-zA-Z0-9]")
echo "RESP:'$RESP'"
echo $body
#echo $wow >> logs.txt
psql -d $database -c "UPDATE tblname SET status='hehe' WHERE mobile='$list'"
done < $filename
Your "$list" JSON entry is not populated with the content of the $list variable because it is within single quotes of the --data-raw curl parameter.
What you need is compose your JSON data for the query before-hand, preferably with the help of jq or some other JSON processor, before sending it as argument to the curl's POST request.
Multiple faults in your scripts (not exhaustive):
Shebang is wrong with a space #! /bin/bash
expireDate=$(date -d "+3 days") return date in locale format and this may not be what you need for your request.
The request and the response data are not processed with JSON grammar aware tools. grep is not appropriate for JSON data.
Some clues but cannot fix your script more without knowing more about the API answers and functions you use.
Anyway here is how you can at least compose a proper JSON request:
#!/usr/bin/env bash
filename='/var/lib/postgresql/Script/list.txt'
database='dbname'
refLink='URL'
authorization='zxc'
expireDate=$(date -R -d "+3 days")
body="Message."
while IFS=' ' read -r list; do
raw_json="$(
jq -n --arg bdy "$body" --arg mobN "$list" \
'.action_type="NotificationActivity"|.title="Expiration Notice"|.type="Notification"|.body=$bdy|.min[0].mobileNumber=$mobN|.'
)"
json_reply="$(curl --location --request POST "$refLink" \
--header "Authorization: Basic $authorization" \
--header 'Content-Type: application/json' \
--data-raw "$raw_json")"
echo "json_reply: '$json_reply'"
echo "$body"
# psql -d "$database" -c "UPDATE tblname SET status='hehe' WHERE mobile='$list'"
done <"$filename"
I am trying to pass values with spaces in curl POST method. I am directing the values through a txt file. POST command does not allow me to pass values with spaces using the for while loop, But when i pass it without while loop it accepts the value without any error.
Below are the commands
This works perfectly fine
curl -d '{"name": "equity calculation support", "email": "email#test.com"}' -H "Authorization: Basic YWRtaW46YWRtaW4=" -H "Content-Type: application/json" -H "Accept: application/json" -X POST http://localhost:3000/api/teams
{"message":"Team created","teamId":103}
when using while loop and IFS it doesn't take the values with spaces:
while IFS= read -r line ; do curl -d '{"name": "'$line'"}' -H "Authorization: Basic YWRtaW46YWRtaW4=" -H "Content-Type: application/json" -H "Accept: application/json" -X POST 'http://localhost:3000/api/teams'; done < /tmp/group.txt
group.txt file contains the values .
You aren't quoting the expansion of $line:
while IFS= read -r line ; do
curl -d '{"name": "'"$line"'"}' \
-H "Authorization: Basic YWRtaW46YWRtaW4=" \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-X POST 'http://localhost:3000/api/teams'
done < /tmp/group.txt
However, it's a better idea to let a tool like jq produce the JSON, to ensure that any characters in $line that need to be escaped to produce proper JSON do, indeed, get escaped.
while IFS= read -r line; do
d=$(jq -n --argjson x "$line" '{name: $x}')
curl -d "$d" ...
done < /tmp/group.txt
It looks like the JSON you want to create would fit on a single line, so you could also process all of /tmp/group.txt with a single call to jq, and pipe its output to your loop.
jq -c -R '{name: .}' | while IFS= read -r line; do
curl -d "$line" ...
done
I am trying to make a script to use POST instead of GET but I not success at it.
This is the script:
#!/usr/bin/bash
echo "Content-type: application/json"
echo ""
login=`echo "$QUERY_STRING" | awk '{split($0,array,"&")} END{print array[1]}' | awk '{split($0,array,"=")} END{print array[2]}'`
clave=`echo "$QUERY_STRING" | awk '{split($0,array,"&")} END{print array[2]}' | awk '{split($0,array,"=")} END{print array[2]}'`
if curl -H "Content-Type: application/json" -d "username="$login"&passcode="$clave"" -X POST http://x.x.x.x/abc -s | grep -q mnp;
then
echo 1
else
echo 3
fi
the script works but it still using GET.
screenshot of HTTP request
Any ideas about the right format of the Curl command?
Regards
I have txt file like this:
1 a
2 b
3 c
I want to take these datas step by step for example first ı will get " 1 " and put it a varible and then get " a " put it in a varible and run a curl command
I mean first first row first column then fisrt row second column then second row first column .....
ı wrote a script in below but not working, it turns null varible for b and c
#!/bin/sh
for ((i=1;i<=4;i++)); do
echo $i
b=$(awk 'NR==$i { print $1}' a.txt)
c=$(awk 'NR==$i { print $2}' a.txt)
echo $b
echo $c
curl -X POST \
-H "X-netmera-api-key: xxxxxxxxxx" \
-H "Content-Type: application/json" \
-d '[
{
"deviceToken" : "$b",
"platform" : "1",
"extId" : "$c"
}
]' \
https://xxxx.xxxxx.com/rest/3.0/xxxxx
done
Throw awk away; it isn't necessary here.
while read -r b c; do
curl -X POST \
-H "X-netmera-api-key: xxxxxxxxxx" \
-H "Content-Type: application/json" \
-d "[
{
\"deviceToken\" : \"$b\",
\"platform\" : \"1\",
\"extId\" : \"$c\"
}
]" \
https://xxxx.xxxxx.com/rest/3.0/xxxxx
done < a.txt
You shouldn't be trying to generate JSON dynamically like this, but that's an issue for another question.
You will need to pass variables to awk with -v and so:
b=$(awk -v varb=$i 'NR==varb { print $1}' a.txt)
Here we are setting the awk variable varb equal to the i variable of your bash loop.
I have a simple bash script that uses an api to add itself into a database. But the script keeps adding ' ' to my variables and its breaking curl.
hostname=`hostname`
ip_address=`ip add show eth0 | grep 'inet ' | awk '{ print $2 }' | cut -d '/' -f1`
env=`hostname | cut -d '-' -f1`
os=`cat /etc/issue.net | head -1`
date=`date`
curl -H 'Content-Type: application/json' -PUT "https://10.10.10.10/database" -k -d '{"Environment":"'$env'","Hostname":"'$hostname'","IP_Address":"'$ip_address'","OS":"'$os'","State":"OK","Updated_Time":"'$date'"}'
exit $?
The output is this:
curl -H 'Content-Type: application/json' -PUT https://10.10.10.10/database -k -d '{"Environment":"ops","Hostname":"ex-example-host","IP_Address":"10.10.10.10","OS":"Ubuntu' 14.04 'LTS","State":"OK","Updated_Time":"Thu' Aug 14 15:27:55 PDT '2014"}'
Both the $date and $hostname put ' ' on the inside the format breaking the curl. Is there a way to fix this?
Thanks,
The problem is that you are leaving the parameter expansions unquoted in bash, so spaces in those values break up the word being passed to curl. It would be simpler to swap your use of double and single quotes, if JSON allowed you to use single quotes. That not being the case, I'd store the JSON in a variable using read and a here document first to simplify the quoting.
read -r data <<EOF
{"Environment":"$env","Hostname":"$hostname","IP_Address":"$ip_address","OS":"$os","State":"OK","Updated_Time":"$date"}
EOF
curl ... -d "$data"