Subscribe and Publish in BASH like a receive Buffer - bash

I want to buffer all messages of a subscription during processing publish messages.
The reason is a firmware update over MQTT. I want to send data with crc and the microcontroller have to verify the content and need to answer with new address pointer.
For now, i got it working without any interaction from the microcontroller. If size doesn't fit, or mc is not online, my bash script sends all the data of the binary file into the nirvana.
If I publish a message and subscribe directly after the publish, it is possible to lose an answer very easily.
But I'm not so experienced with bash and I need some suggestions how to do this...
This is what I have:
mosquitto_pub -h $MQTT_SERVER -t "spiffs/${client_name}/file_info" -m "$binary_file" -p 8883 -i "fw_server" --cafile server-cert.pem -d --insecure -u $MQTT_USER -P $MQTT_PW
mosquitto_sub -h $MQTT_SERVER -p 8883 -t "+/${client_name}/#" -i "fw_server" --cafile server-cert.pem -d --insecure -u $MQTT_USER -P $MQTT_PW | while read -r payload
do
fp_jq_rspCode=".upload.fp" # file pointer
address=$(echo $payload | jq -r $fp_jq_rspCode)
echo "${payload}" | jq -c '.[]'
echo "address pointer:$address"
c=${address}
addr=$(printf '%08x' "$c")
content=$(xxd -p -c $payload_size -l $payload_size -seek $c $binary_file)
length=`expr length "$content"`
len=$(printf '%04x' "$length")
crc32=$(echo -n "$content" | gzip -c | tail -c8 | hexdump -n4 -e '"%u"')
crc32=$(printf '%08x' "$crc32")
echo "$addr $len $content $crc32"
message=$(printf '%08x%04x%s%s' "$c" "$length" "$content" "$crc32")
mosq=$(mosquitto_pub -h $MQTT_SERVER -t "spiffs/${client_name}/upload" -m "$message" -p 8883 -i "fw_server" --cafile server-cert.pem -d --insecure -u $MQTT_USER -P $MQTT_PW)
done

Related

Sending grep data iteratively using curl for output of tail -f

I am using grep to extract data from a log file. The log file is dynamically updating new rows. and I need to send the grepped data to a REST endpoint using curl. This can be done easily for a static file but cannot find a solution fo a running log file. How can I realize this situation?
eg: tail -f | grep "<string>" > ~/<fileName>.log
The above can put the data in a file. Need to send it using a POST curl.
Maybe using a function like
send_data(){
curl -s -k -X POST –header Content-Type: application/json’ \
–header ‘Accept: application/json’ \
“http://${HOST}${PORT}/v1/notify” \
-d $1
}
If tail -f | grep "<string>" > ~/<fileName>.log is working for you then you could do:
tail -f file | stdbuf -i0 -o0 -e0 grep "<string>" | xargs -n 1 -d $'\n' curl ...
or:
while IFS= read -r line; do
curl ... "$line"
done < <(tail -f file | stdbuf -i0 -o0 -e0 grep "<string>")

POST using Bash in Script

I am currently writing a script which gets a cookie and unique URL from a website, then uses that cookie and URL to post login/password information.
I am having trouble in getting this to work, some of the runs I have done gives me the message "switching from POST to GET" and I can't understand why.
Can someone please help me?
for i in $(cat accounts.txt); do
rm out.out
curl -L -b cookies.txt -c cookies.txt https://website.com | grep -Eo "(http|https)://cookies.website.com/idp[a-zA-Z0-9./?=_-]*" >> out.out
url=`head -1 out.out`
content="$(curl -X POST -L -v -b cookies.txt -d "$i" "$url" )"
echo "$url" "$i" "$content"
done

How Extract the http code, size header and title from website in bash?

With curl I'm extracting the %{http_code} %{size_header} %{redirect_url} and it works great but, I also like to add the website title to the Bash response.
usage: check-website.sh website-list.txt
#!/bin/bash
FILE="$1"
while read LINE; do
curl -H 'Cache-Control: no-cache' -i -s -k -o /dev/null --silent --max-time 2 --write-out '%{http_code} %{size_header} %{redirect_url}' "$LINE"
echo " $LINE"
done < ${FILE}
I was looking on extracting the titles with this curl command:
curl -s example.com | grep -o "<title>[^<]*" | cut -d'>' -f2-
but I'm not sure on how to do this, in order to get the bash output like:
%{http_code} %{size_header} %{redirect_url} $website_title
Can anyone help me to join the title?
Instead of redirecting to /dev/null you can redirect to your grep command. Note that you should use the case insensitive grep option. I also added an extra space at the end of your --write-out:
curl -H 'Cache-Control: no-cache' -i -s -k -o >(grep -io "<title>[^<]*" | cut -d'>' -f2-) --silent --max-time 2 --write-out '%{http_code} %{size_header} %{redirect_url} ' "$LINE"

bash script read server and not read password

I use curl to open with username and password router
curl http://192.168.1.1 --user admin:admin |grep -i "stats"
But when I made this code to use curl from bash script I have problem to read server and password from file `LINKS_FILE="server"
PASS="passwd"
for link in `cat "$LINKS_FILE"`
do
for pass in `cat "$PASS"`
do
res=$(curl -m 1 "http://${link}:8080" --user admin:${pass} )
if echo $res | grep -i "stats"; then `

Insert string after match in variable

I am trying to make some workaround to solve a problem.
We have a gtk+ program that call a bash script who calls rdesktop.
In a machine, we discover that the rdesktop call need on extra parameter...
Since i didnt write anything of this code, and i can modify the GTK part of the problem, i can only edit the bash script that make the middle call between the calls.
i have a variable called CMD with something that look like:
rdesktop -x m -r disk:USBDISK=/media -r disk:user=/home/user/ -r printer:HP_Officejet_Pro_8600 -a 16 -u -p -d -g 80% 192.168.0.5
i need to "live edit" this line for when the printer parameter exists, it append ="MS Publisher Imagesetter" after the printer name.
The best i accompplish so far is
ladb#luisdesk ~ $ input="rdesktop -x m -r disk:USBDISK=/media -r disk:user=/home/user/ -r printer:HP_Officejet_Pro_8600 -a 16 -u -p -d -g 80% 192.168.0.5"
ladb#luisdesk ~ $ echo $input | sed s/'printer:.*a /=\"MS Publisher Imagesetter\" '/
Which return me:
rdesktop -x m -r disk:USBDISK=/media -r disk:user=/home/user/ -r ="MS Publisher Imagesetter" 16 -u -p -d -g 80% 192.168.0.5
Almost this, but i need to append the string, not replace it.
help?
Edit: i pasted incomplete exemples. fixed
Edit2:
With the help of who respond, i end up with
echo "$input" | sed 's/\(printer:\)\([^ ]*\)/\1\2="MS Publisher Imagesetter"/'
If you want the output to look like:
rdesktop -x m -r disk:USBDISK=/media -r disk:user=/home/user/ -r printer:"HP_Officejet_Pro_8600 MS Publisher Imagesetter" -a 16 -u -p -d -g 80% 192.168.0.5
This sed will do, it matches the printer: part first then the existing printer name and quotes both, if not you can adjust the replacement
variables to put the quotes/spacing where you want:
input="rdesktop -x m -r disk:USBDISK=/media -r disk:user=/home/user/ -r printer:HP_Officejet_Pro_8600 -a 16 -u -p -d -g 80% 192.168.0.5"
echo "$input" | sed 's/\(printer:\)\([^ ]*\)/\1"\2 MS Publisher Imagesetter"/'
output:
rdesktop -x m -r disk:USBDISK=/media -r disk:user=/home/user/ -r printer:"HP_Officejet_Pro_8600 MS Publisher Imagesetter" -a 16 -u -p -d -g 80% 192.168.0.5
You can use this:
sed 's/printer:[^=]\+=/\0 "MS Publisher Imagesetter"/' <<< "$input"
The \0 in the replacement pattern outputs the match itself.

Resources