Timer in a loop - Bash - bash

I have a token whos expire in 30 minutes, i want do renew this token until the 30 minutes time expire, i have the following script calling an api:
function token {
gen_bearer_token=$(curl -X POST "https://api.foo.bar/oauth2/token" -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "client_id=foo&client_secret=bar" | cut -d '{' -f2 | cut -d '}' -f1)
bearer_token=$(echo $gen_bearer_token | awk '{print $2}' | cut -d '"' -f2 | cut -d '"' -f1)
token_type=$(echo $gen_bearer_token | awk '{print $6}' | cut -d '"' -f2 | cut -d '"' -f1)
}
echo -e "HOSTNAME;LAST_SEEN" > ${file}
ids=$(curl -s -X GET 'https://api.foo.bar/devices/queries/devices-scroll/v1?limit=5000' -H 'accept: application/json' -H 'authorization: '${token_type}' '${bearer_token}'' | jq .resources[])
for id in ${ids}
do
result=$(curl -s -X GET 'https://api.foo.bar/devices/entities/devices/v1?ids='${id}'' -H 'accept: application/json' -H 'authorization: '${token_type}' '${bearer_token}'' | jq '.resources[] | "\(.hostname) \(.last_seen)"')
if [[ ! -z ${result} ]]
then
hostname=$(echo ${result} | awk '{print $1}' | cut -d '"' -f2)
last_seen=$(echo ${result} | awk '{print $2}' | cut -d '"' -f1)
echo -e "${hostname};${last_seen}" >> ${file}
fi
done
in this looping, time duration is more than 2 hours (variable duration) + i want to create a timer to renew token if the times pass through 30 minutes, if passed 30 minutes the request in api will fail.
for id in ${ids}
do
result=$(curl -s -X GET 'https://api.foo.bar/devices/entities/devices/v1?ids='${id}'' -H 'accept: application/json' -H 'authorization: '${token_type}' '${bearer_token}'' | jq '.resources[] | "\(.hostname) \(.last_seen)"')
if [[ ! -z ${result} ]]
then
hostname=$(echo ${result} | awk '{print $1}' | cut -d '"' -f2)
last_seen=$(echo ${result} | awk '{print $2}' | cut -d '"' -f1)
echo -e "${hostname};${last_seen}" >> ${file}
fi
done

bash as a suitable timer built-in: the SECONDS variable.
Its value is its initial value plus the number of seconds since the most recent assignment. At shell start-up, it is initialized to 0.
All you need to do is check the value of $SECONDS before using the token, and if its value is greater than 1800 (the number of seconds in 30 minutes), get a new one.
SECONDS=1801 # Make sure a token will be generated before the first call
for id in $ids; do
if [ "$SECONDS" -gt 1800 ]; then
token
SECONDS=0
fi
...
done

I Solved this problem creating a check response with status code.
response=$(curl -H 'authorization: '${token_type}' '${bearer_token}'' --write-out '%{http_code}' --silent --output /dev/null https://api.foo.bar/devices/entities/devices/v1?ids='0000000000000')
if [ ${response} -eq 200 ]; then
do something
else
token
fi

Related

why does this for loop only loop once?

what i'm trying to do is a simple practice code with a output like this:
user1 logged in at hh:mm
user2 logged in at hh:mm
...
so i wrote a basic for loop, but what i don't understand why it only runs once.
for user in $(who)
do
USERID=$(echo $user | tr -s ' ' | cut -d' ' -f1)
LOGTIME=$(echo $user | tr -s ' '| cut -d' ' -f4)
echo "$USERID logged in at $LOGTIME"
done
this echos the following:
user1
user2
user3 logged in at hh:mm
hh:mm
hh:mm
i've put echo lines in between to figure out what is going on and how to fix it, but the only thing i figured out is that the for loop is only running once so USERID and LOGTIME is just a long string with linebreaks. what am i doing wrong? (i did the IFS=$ā€™\nā€™)
Would you please try the following:
while read -r user tty mon day logtime; do
echo "$user logged in at $logtime"
done < <(who)
#!/bin/sh
who > stack
while [ $(wc -l stack) -gt 0 ]
do
line=$(sed '1p' stack)
USERID=$(echo ${line} | tr -s ' ' | cut -d' ' -f1)
LOGTIME=$(echo ${line} | tr -s ' '| cut -d' ' -f4)
echo "${USERID} logged in at ${LOGTIME}"
sed -i '1d' stack
done
rm -v ./stack
exit 0
If this does not work, provide me with an example of who output on your machine.

Send a burst/multi threaded requests using bash and curl

I have a use case where I want to send a burst of requests. My current script sends the request sequentially but I want to be able to send a burst of requests (all of them at once) and wait for the response to come back (ex. if I enter 10 as request count- all 10 should be sent together). I am not sure how I can achieve that using bash and curl. Any ideas would be appreciated.
#!/bin/bash
CURL="/usr/bin/curl"
echo -n "how many times you want to run the request: "
read erc
ERC="$erc"
#count=1;
total_connect=0
total_start=0
total_time=0
echo " Time_Connect Time_startTransfer Time_total HTTP_Code ";
#while [ $count -le $ERC ]
for ((i=1;i<=$ERC;i+=1)); do
result=`$CURL -k -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}:%{http_code} -H "Accept: application/xml" -H "Accept: text/xml" -H "Accept: application/json" -H "Accept: application/cbor" "http://google.com"`
echo $result;
var=$(echo $result | awk -F":" '{print $1, $2, $3, $4}')
set -- $var
total_connect=`echo "scale=6; $total_connect + $1" | bc`;
total_start=`echo "scale=6; $total_start + $2" | bc`;
total_time=`echo "scale=6; $total_time + $3" | bc`;
#count=$((count+1))
done
echo "URL executed is http://google.com"
echo "HTTP CODE of all request is $4"
echo "average time connect: `echo "scale=6; $total_connect/$ERC" | bc`";
echo "average time start: `echo "scale=6; $total_start/$ERC" | bc`";
echo "average Totaltime taken: `echo "scale=6; $total_time/$ERC" | bc`";
Here's the code with the & introduced to issue all of the curls "at the same time".
#!/bin/bash
CURL="/usr/bin/curl"
echo -n "how many times you want to run the request: "
read erc
ERC="$erc"
#count=1;
total_connect=0
total_start=0
total_time=0
echo " Time_Connect Time_startTransfer Time_total HTTP_Code ";
#while [ $count -le $ERC ]
for ((i=1;i<=$ERC;i+=1)); do
{
result=`$CURL -k -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}:%{http_code} -H "Accept: application/xml" -H "Accept: text/xml" -H "Accept: application/json" -H "Accept: application/cbor" "http://google.com"`
echo $result;
var=$(echo $result | awk -F":" '{print $1, $2, $3, $4}')
set -- $var
total_connect=`echo "scale=6; $total_connect + $1" | bc`;
total_start=`echo "scale=6; $total_start + $2" | bc`;
total_time=`echo "scale=6; $total_time + $3" | bc`;
#count=$((count+1))
} > $i.result &
done
wait
# TODO: You'll need to write code here to process the intermediate results in *.result
# It would have been helpful to have created a temporary directory for your indivdual results as well
echo "URL executed is http://google.com"
echo "HTTP CODE of all request is $4"
echo "average time connect: `echo "scale=6; $total_connect/$ERC" | bc`";
echo "average time start: `echo "scale=6; $total_start/$ERC" | bc`";
echo "average Totaltime taken: `echo "scale=6; $total_time/$ERC" | bc`";
The wait I introduced tells bash to wait for all of the child processes (curls and such) to complete before continuing.

curl sends empty json on content-type: application/json

I am sending cpu and memory usage on json format from Ubuntu to Node.js API using curl with POST method. However the json data is empty at Node.js server.
Bash script on Ubuntu
top -b -n 2 -d 0.5 > top.txt
cpu_i=$(grep Cpu top.txt | cut -d ',' -f 4 | cut -d ' ' -f 2)
cpu=$(echo $cpu_i | cut -d ' ' -f 2)
echo $cpu
mem=$(grep "KiB Mem :" top.txt | cut -d ':' -f 2)
#echo $mem
mem_used=$(echo $mem | cut -d ',' -f 3 | cut -d ' ' -f 2)
echo $mem_used
curl --header "Content-Type: application/json" -d "{\"cpu\":\"$cpu\", \"memory\":\"$mem_used\",\"device\":\"ubuntu\"}" http://192.168.10.10:4000/collector
Output at Node.js server
{}
Remote Address: ::ffff:192.168.10.5
If I remember correctly, the httpverb is set to get by default. If you intend to post it, use -X POST. That will probably solve your problem, because the curl command is OK.
curl -X POST --header "Content-Type: application/json" -d "{\"cpu\":\"$cpu\", \"memory\":\"$mem_used\",\"device\":\"ubuntu\"}" http://192.168.10.10:4000/collector

Strange behavior when parsing result from curl + awk

Using curl on ubuntu I am trying to fetch the Jenkins version inspired by:
https://wiki.jenkins.io/display/JENKINS/Remote+access+API
In a bash script I do:
VERSION=$(curl -k -i -X GET --insecure --silent --header \"Authorization: Bearer $TOKEN \" $URL | grep -Fi X-Jenkins: | awk '{print $2}')
echo "__A__[${VERSION}]__B__"
But when I run the script I get:
]__B__2.89.2
So for some reason the prefix: __A__[ gets swallowed and the suffix gets turned into a prefix.
I have also tried to trim the output with:
VERSION=$(curl -k -i -X GET --insecure --silent --header \"Authorization: Bearer $TOKEN \" $URL | grep -Fi X-Jenkins: | awk '{print $2}' | sed -e 's/^[ \t]*//')
But it gives the same result.
As suggested below I have also tried with:
echo '__A__['"${VERSION}"']__B__'
But still gives the same/wrong result.
A few other things I have tried (giving the same result)
Same/wrong output
VERSION=$(curl -k -i -X GET --insecure --silent --header \"Authorization: Bearer $TOKEN \" $URL | grep -i X-Jenkins: | awk '{print $2}')
echo '__A__['"${VERSION}"']__B__'
Same/wrong output
VERSION=$(curl -k -i -X GET --insecure --silent --header \"Authorization: Bearer $TOKEN \" $URL | grep X-Jenkins: | awk '{print $2}')
echo '__A__['"${VERSION}"']__B__'
Based on below suggestion I have now tried:
echo $VERSION|od -ax
Which gives:
0000000 2 . 8 9 . 2 cr nl
2e32 3938 322e 0a0d
0000010
If I compare that with:
VERSION_TEMP="2.89.2"
echo $VERSION_TEMP|od -ax
I get:
0000000 2 . 8 9 . 2 nl
2e32 3938 322e 000a
0000007
So looks like its the cr in the VERSION var that is causing the issue (not sure how that explains the whole reversing of prefix/suffix as described above).
SOLVED: Based on input from Romeo I now got it to work with adding |tr -d '\r' :
VERSION=$(curl -k -i -X GET --insecure --silent --header \"Authorization: Bearer $TOKEN \" $URL | grep X-Jenkins: | awk '{print $2}'|tr -d '\r')
Apparently the output contains a DOS carriage return.
Try adding tr -d '\015':
version=$(curl -k -i -X GET --insecure --silent --header \"Authorization: Bearer $TOKEN \" "$URL" |
tr -d '\015' |
awk 'tolower($0) !~ /x-jenkins:/{print $2}')
echo "__A__[$version]__B__"
Uppercase variable names are reserved for system use, so I changed yours to lower case, too, and removed the useless grep.

Can not to use POST in CURL

I am trying to make a script to use POST instead of GET but I not success at it.
This is the script:
#!/usr/bin/bash
echo "Content-type: application/json"
echo ""
login=`echo "$QUERY_STRING" | awk '{split($0,array,"&")} END{print array[1]}' | awk '{split($0,array,"=")} END{print array[2]}'`
clave=`echo "$QUERY_STRING" | awk '{split($0,array,"&")} END{print array[2]}' | awk '{split($0,array,"=")} END{print array[2]}'`
if curl -H "Content-Type: application/json" -d "username="$login"&passcode="$clave"" -X POST http://x.x.x.x/abc -s | grep -q mnp;
then
echo 1
else
echo 3
fi
the script works but it still using GET.
screenshot of HTTP request
Any ideas about the right format of the Curl command?
Regards

Resources