I have a problem with this code. It states that there is a URL issue but I don't see that as right. I am not responsible for the server though. I just wanted to test some IoT properties with basic linux commands.
#!/bin/bash
a=Client secret
c=Custom domain
b=Client id:
#!/bin/bash
response=$(curl -X POST --user $b:$a "https://${c}.auth.eu-central-1.amazoncognito.com/oauth2/token?grant_type=client_credentials" -H 'Content-Type: application/x-www-form-urlencoded')
token=$(jq -r '.access_token' <<< "$response")
secondsValid=$(jq -r '.expires_in' <<< "$response")
refreshToken=$(jq -r '.refresh_token' <<< "$response")
for i in {1..4}
do
declare -i TIME
TIME=$(date +%s)
TEMP=$(sensors -j | jq '."cpu_thermal-virtual-0"."temp1"."temp1_input"')
curl -i -k -X POST -H "Authorization: $token"-H "Content-Type: application/json" --data '{ "id":"2","timestamp":"'${TIME}'","data":"'${TEMP}'"}' https://dv7knsjzph.execute-api.eu-central-1.amazonaws.com/prod/boxtronic-devices/2/data/
sleep 1
echo "$refreshToken"
echo "$secondsValid"
echo "$TIME"
done
Anyone knows why I am getting the error ?
I have a requirement where my curl command is receiving continuous output from a streaming HTTP service. The stream never ends. I want to just grep a string from the stream and pass/pipe this command output to another utility such as xargs and say, echo for an example, for further continuous processing.
This is the output of the continuous stream which I shall stop receiving only when I end running the curl command.
curl -X "POST" "http://localhost:8088/query" --header "Content-Type: application/json" -d $'{"ksql": "select * from SENSOR_S EMIT CHANGES;","streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"}}' -s -N
[{"header":{"queryId":"none","schema":"`ROWTIME` BIGINT, `ROWKEY` STRING, `SENSOR_ID` STRING, `TEMP` BIGINT, `HUM` BIGINT"}},
{"row":{"columns":[1599624891102,"S2","S2",40,20]}},
{"row":{"columns":[1599624891113,"S1","S1",90,80]}},
{"row":{"columns":[1599624909117,"S2","S2",40,20]}},
{"row":{"columns":[1599624909125,"S1","S1",90,80]}},
{"row":{"columns":[1599625090320,"S2","S2",40,20]}},
Now when I pipe the output to grep, it works as expected and I keep receiving any new events.
curl -X "POST" "http://localhost:8088/query" --header "Content-Type: application/json" -d $'{"ksql": "select * from SENSOR_S EMIT CHANGES;","streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"}}' -s -N | grep S1
{"row":{"columns":[1599624891113,"S1","S1",90,80]}},
{"row":{"columns":[1599624909125,"S1","S1",90,80]}},
But when I pipe this grep output to xargs and echo, the output just don't move at all.
curl -X "POST" "http://localhost:8088/query" --header "Content-Type: application/json" -d $'{"ksql": "select * from SENSOR_S EMIT CHANGES;","streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"}}' -s -N | grep S1 | xargs -I {} echo {}
^C
When I remove grep from the middle, it works as expected.
curl -X "POST" "http://localhost:8088/query" --header "Content-Type: application/json" -d $'{"ksql": "select * from SENSOR_S EMIT CHANGES;","streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"}}' -s -N | xargs -I {} echo {}
[{header:{queryId:none,schema:`ROWTIME` BIGINT, `ROWKEY` STRING, `SENSOR_ID` STRING, `TEMP` BIGINT, `HUM` BIGINT}},
{row:{columns:[1599624891102,S2,S2,40,20]}},
{row:{columns:[1599624891113,S1,S1,90,80]}},
{row:{columns:[1599624909117,S2,S2,40,20]}},
{row:{columns:[1599624909125,S1,S1,90,80]}},
{row:{columns:[1599625090320,S2,S2,40,20]}},
Looks like grep is looking for the input to end before it can pipe it further. When I tested the same thing with a finite input, it works as expected.
ls | grep sh | xargs -I {} echo {};
abcd.sh
123.sh
pqr.sh
xyz.sh
So, the questions are: Is my understanding correct? Is there a way where grep can keep passing the output to subsequent commands in real time? I want to keep some basic filtering logic out of the further scripting, hence wanting grep to work.
Thanks in Advance !
Anurag
As suggested by #larsks , " --line-buffered flush output on every line" option for grep is working fine when is test for similar requirement as yous .
So the command would be
curl -X "POST" "http://localhost:8088/query" --header "Content-Type: application/json" -d $'{"ksql": "select * from SENSOR_S EMIT CHANGES;","streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"}}' -s -N | grep S1 --line-buffered | xargs -I {} echo {}
I tested on "/var/log/messages" file which gets continously updated as following :
[root#project1-master ~]# tail -f /var/log/messages | grep journal --line-buffered | xargs -I {} echo {}
Sep 11 11:15:47 project1-master journal: I0911 15:15:47.448254 1 node_lifecycle_controller.go:1429] Initializing eviction metric for zone:
Sep 11 11:15:52 project1-master journal: I0911 15:15:52.448704 1 node_lifecycle_controller.go:1429] Initializing eviction metric for zone:
Sep 11 11:15:54 project1-master journal: 2020-09-11 15:15:54.006 [INFO][46] felix/int_dataplane.go 1300: Applying dataplane updates
I'm attempting to use the Spotify web API to get the currently playing track information to display on a conky widget without the need for the native Spotify player. So far I have managed to reliably get the authorization key, but that's about as far as I've been able to figure out, many dozens of examples, tutorials, and attempts later I'm still stuck.
How would I go about getting the access token, and refreshing it for that matter?
This is the script that works so far
client_id=$(jq -r '.id' ./spotify_creds.json)
client_secret=$(jq -r '.secret' ./spotify_creds.json)
port=8082
if [ -f "./auth_key" ]; then
echo "auth key found"
else
redirect_uri=http%3A%2F%2Flocalhost%3A$port%2F
auth_endpoint=https://accounts.spotify.com/authorize/?response_type=code\&client_id=$client_id\&redirect_uri=$redirect_uri
scopes="user-read-currently-playing"
encoded_scopes=$(echo $scopes| tr ' ' '%' | sed s/%/%20/g)
xdg-open $auth_endpoint\&scope=$encoded_scopes
(echo "HTTP/1.1 200 OK\nAccess-Control-Allow-Origin:*\nContent-Length:65\n\n<html><script>open(location, '_self').close();</script></html>\n" | nc -l $port -q 0 > "auth_key")
fi
code=$(echo $(cat ./auth_key) | grep GET | cut -d' ' -f 2 | cut -d'=' -f 2)
echo " authentication code"
echo $code
some things I have tried for getting the token that did not work are:
encoded_creds="$(echo -n "$client_id:$client_secret" | base64)"
(curl -X POST -s https://accounts.spotify.com/api/token \
-H "Content-Type:application/x-www-form-urlencoded" \
-H "Authorization: Basic $encoded_creds" \
-d "grant_type=authorization_code&code=$code&redirect_uri=http%3A%2F%2Flocalhost%3A$port%2F") > "auth_token"
and
encoded_creds="$(echo -n "$client_id:$client_secret" | base64)"
(curl -X POST "https://api.spotify.com/v1/swap" -H "Content-Type: application/x-www-form-urlencoded" --data "code=$code") > "auth_token"
along with many other similar variations, all either returning nothing, or a 401 error that no token was provided.
I am sending cpu and memory usage on json format from Ubuntu to Node.js API using curl with POST method. However the json data is empty at Node.js server.
Bash script on Ubuntu
top -b -n 2 -d 0.5 > top.txt
cpu_i=$(grep Cpu top.txt | cut -d ',' -f 4 | cut -d ' ' -f 2)
cpu=$(echo $cpu_i | cut -d ' ' -f 2)
echo $cpu
mem=$(grep "KiB Mem :" top.txt | cut -d ':' -f 2)
#echo $mem
mem_used=$(echo $mem | cut -d ',' -f 3 | cut -d ' ' -f 2)
echo $mem_used
curl --header "Content-Type: application/json" -d "{\"cpu\":\"$cpu\", \"memory\":\"$mem_used\",\"device\":\"ubuntu\"}" http://192.168.10.10:4000/collector
Output at Node.js server
{}
Remote Address: ::ffff:192.168.10.5
If I remember correctly, the httpverb is set to get by default. If you intend to post it, use -X POST. That will probably solve your problem, because the curl command is OK.
curl -X POST --header "Content-Type: application/json" -d "{\"cpu\":\"$cpu\", \"memory\":\"$mem_used\",\"device\":\"ubuntu\"}" http://192.168.10.10:4000/collector
I am trying to execute curl using this shell script:
#!/bin/bash
curl -k -H "Content-Type:application/json" -d '{"username":"admin","password":"adminpw", "tenant":"master"}' https://localhost/tron/api/v1/tokens > /tmp/token.data
grep -Po '{\"token\":\"\K[^ ]...................' /tmp/token.data > /tmp/token
tokendetails=`cat /tmp/token`
for token in $tokendetails
do
TOKEN=`echo $token`
done
userdetails=`cat /tmp/curloutput.txt | sed 's/{"clientInactivityTime"/\n{"clientInactivityTime"/g' | sed 's/\(.*\).\("firstName":[^,]*\)\(.*\)\("lastName":[^,]*\)\(.*\)\("email":[^,]*\)\(.*\)\("username":[^,]*\)\(.*\)/\2,\4,\6,\8/g' | grep username`
for user in $userdetails
do
firstName=`echo $user | sed 's/,/\n/g' | grep firstName | sed 's/.*:"\([^"]*\).*/\1/g'`
lastName=`echo $user | sed 's/,/\n/g' | grep lastName | sed 's/.*:"\([^"]*\).*/\1/g'`
email=`echo $user | sed 's/,/\n/g' | grep email | sed 's/.*:"\([^"]*\).*/\1/g'`
username=`echo $user | sed 's/,/\n/g' | grep username | sed 's/.*:"\([^"]*\).*/\1/g'`
curl -k -X POST "https://haxsne09/tron/api/v1/users" -H "accept: application/json" -H "Authorization: Bearer =${TOKEN}" -H "Content-Type: application/x-www-form-urlencoded" -d "first_name=${firstName}\&last_name=${lastName}\&email=${email}\&password=Tata123^\&username=${username}\&is_active=true"
echo $RESPONSE
done
I am getting ths error:
{"Error":"bpapigw-300 Cannot authorize access to resource: Could not authorize path for user identifier: Failed to get Roles for identifier: REST operation failed 0 times: '[GET /api/v1/current-user][401] currentUserListUnauthorized \u0026{Detail:Invalid token}'. This user is unauthenticated?"}
Do I need to add any syntax before executing curl -k -X POST?
What I see is that -H "Authorization: Bearer =${TOKEN}" contains an = sign which shouldn't be there...
It should be: -H "Authorization: Bearer ${TOKEN}"
More, in a command you use /tmp/curloutput.txt file, which is never created by your script...
The Authorization header you are using is not working. Maybe the syntax is not Bearer =aAbBcCdDeEfF0123456 but something else for the server running on haxsne09, maybe without the = like #MarcoS suggests.
Alternatively, your grep command may be returning one too many characters (a rogue quote maybe).
I rewrote your code below to be more readable. You will notice that I:
Changed your matching groups in sed to capture only the needed parts and put them in variables using read. I also used the -E flag to avoid having to use \( and \)
Removed the useless for loops
Quoted all variable expansions properly
Added some line breaks for readability
Removed some temporary files and associated useless uses of cat
Here is the updated script:
#!/bin/bash
curl -k -H 'Content-Type:application/json' -d \
'{"username":"admin","password":"adminpw", "tenant":"master"}' \
https://localhost/tron/api/v1/tokens > /tmp/token.data
token=$(grep -Po '{"token":"\K[^ ]...................' /tmp/token.data)
IFS=, read -r firstName lastName email username < <(
</tmp/curloutput.txt sed 's/{"clientInactivityTime"/\n&/' |
sed -nE 's/.*."firstName":"([^"]*)".*"lastName":"([^"]*)").*"email":"([^"]*).*"username":"([^"]*)".*/\1,\2,\3,\4/p'
)
curl -k -X POST 'https://haxsne09/tron/api/v1/users' -H 'accept: application/json' \
-H "Authorization: Bearer $token" -H "Content-Type: application/x-www-form-urlencoded" -d \
"first_name=$firstName&last_name=$lastName&email=$email&password=Tata123^&username=$username&is_active=true"
echo