I'm looking small rest server for sent request and execute scenario.
I've found it here:
Create a minimum REST Web Server with netcat nc
I'm trying to execute this small rest server.
Below Dockerfile and bash script.
Dockerfile
FROM debian
ADD ./rest.sh /rest.sh
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
&& apt-get install -y net-tools netcat curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& chmod +x /rest.sh
EXPOSE 80
CMD /rest.sh
rest.sh
#!/bin/bash
/bin/rm -f out
/usr/bin/mkfifo out
trap "rm -f out" EXIT
while true
do
/bin/cat out | /bin/nc -l 80 > >( # parse the netcat output, to build the answer redirected to the pipe "out".
export REQUEST=
while read line
do
line=$(echo "$line" | tr -d '[\r\n]')
if /bin/echo "$line" | grep -qE '^GET /' # if line starts with "GET /"
then
REQUEST=$(echo "$line" | cut -d ' ' -f2) # extract the request
elif [ "x$line" = x ] # empty line / end of request
then
HTTP_200="HTTP/1.1 200 OK"
HTTP_LOCATION="Location:"
HTTP_404="HTTP/1.1 404 Not Found"
# call a script here
# Note: REQUEST is exported, so the script can parse it (to answer 200/403/404 status code + content)
if echo $REQUEST | grep -qE '^/echo/'
then
printf "%s\n%s %s\n\n%s\n" "$HTTP_200" "$HTTP_LOCATION" $REQUEST ${REQUEST#"/echo/"} > out
elif echo $REQUEST | grep -qE '^/date'
then
date > out
elif echo $REQUEST | grep -qE '^/stats'
then
vmstat -S M > out
elif echo $REQUEST | grep -qE '^/net'
then
ifconfig > out
else
printf "%s\n%s %s\n\n%s\n" "$HTTP_404" "$HTTP_LOCATION" $REQUEST "Resource $REQUEST NOT FOUND!" > out
fi
fi
done
)
done
docker build -t ncimange .
docker run -d -i -p 80:80 --name ncrest ncimange
docker container ls
IMAGE COMMAND CREATED STATUS PORTS NAMES
ncimange "/bin/sh -c /rest.sh" 8 seconds ago Up 2 seconds 0.0.0.0:80->80/tcp ncrest
docker ps
IMAGE COMMAND CREATED STATUS PORTS NAMES
ncimange "/bin/sh -c /rest.sh" 41 seconds ago Up 34 seconds 0.0.0.0:80->80/tcp ncrest
docker logs ncrest
empty
From host:
curl -i http://127.0.0.1:80/date
curl: (56) Recv failure: Connection reset by peer
From container:
docker exec -it ncrest /bin/bash
netstat -an|grep LISTEN
tcp 0 0 0.0.0.0:41783 0.0.0.0:* LISTEN
curl -i http://127.0.0.1:80/date
curl: (7) Failed to connect to 127.0.0.1 port 80: Connection refused
curl -i http://127.0.0.1:41783/date
curl: (56) Recv failure: Connection reset by peer
How to connect to netcat rest docker container?
You are installing the "wrong" netcat. Debian has two netcat packages: netcat-traditional and netcat-openbsd, and both are slighty different. The netcat package is an alias of netcat-traditional.
For example, in your case your nc command should be nc -l -p 80, because nc -l 80 will only work on netcat-openbsd.
tl;dr: install netcat-openbsd instead of ǹetcat if you wish to use your script unmodified..
Related
I want to buffer all messages of a subscription during processing publish messages.
The reason is a firmware update over MQTT. I want to send data with crc and the microcontroller have to verify the content and need to answer with new address pointer.
For now, i got it working without any interaction from the microcontroller. If size doesn't fit, or mc is not online, my bash script sends all the data of the binary file into the nirvana.
If I publish a message and subscribe directly after the publish, it is possible to lose an answer very easily.
But I'm not so experienced with bash and I need some suggestions how to do this...
This is what I have:
mosquitto_pub -h $MQTT_SERVER -t "spiffs/${client_name}/file_info" -m "$binary_file" -p 8883 -i "fw_server" --cafile server-cert.pem -d --insecure -u $MQTT_USER -P $MQTT_PW
mosquitto_sub -h $MQTT_SERVER -p 8883 -t "+/${client_name}/#" -i "fw_server" --cafile server-cert.pem -d --insecure -u $MQTT_USER -P $MQTT_PW | while read -r payload
do
fp_jq_rspCode=".upload.fp" # file pointer
address=$(echo $payload | jq -r $fp_jq_rspCode)
echo "${payload}" | jq -c '.[]'
echo "address pointer:$address"
c=${address}
addr=$(printf '%08x' "$c")
content=$(xxd -p -c $payload_size -l $payload_size -seek $c $binary_file)
length=`expr length "$content"`
len=$(printf '%04x' "$length")
crc32=$(echo -n "$content" | gzip -c | tail -c8 | hexdump -n4 -e '"%u"')
crc32=$(printf '%08x' "$crc32")
echo "$addr $len $content $crc32"
message=$(printf '%08x%04x%s%s' "$c" "$length" "$content" "$crc32")
mosq=$(mosquitto_pub -h $MQTT_SERVER -t "spiffs/${client_name}/upload" -m "$message" -p 8883 -i "fw_server" --cafile server-cert.pem -d --insecure -u $MQTT_USER -P $MQTT_PW)
done
I have this function on my bash script:
sudo tshark -i eth0 -T fields -e ip.src -e dns.qry.name -Y "dns.qry.name~." -q 1>>log.txt 2>/dev/null &
while true
do
cat log.txt
done
it is capturing ips and domain names in live mode and save them into log file.
how can configure this live mode to be terminated by pressing a key?
Using tee to watch log and send the command to background, then read input to terminate script
tshark -i eth0 -T fields -e ip.src -e dns.qry.name -Y "ip" -q 2>/dev/null | tee log.txt &
read -n1 c && echo "Got key $c"
exit
Note: running the command in a console will terminate it :-p
for testing I try to run the gcloud devserver inside docker with that comment:
sudo /usr/local/gcloud/google-cloud-sdk/bin/java_dev_appserver.sh --disable_update_check --port=8888 --help /app/target/qdacity-war/ 2>&1 | sudo tee /app/logs/devserver.log > /dev/null &
To check if the devserver has started successfully, I use this script:
#!/bin/bash
# This script waits until the port 8888 is open.
SERVER=localhost
PORT=8888
TIMEOUT=180
TIME_INTERVAL=2
PORT_OPEN=1
PORT_CLOSED=0
time=0
isPortOpen=0
while [ $time -lt $TIMEOUT ] && [ $isPortOpen -eq $PORT_CLOSED ];
do
# Connect to the port
(echo > /dev/tcp/$SERVER/$PORT) >/dev/null 2>&1
if [ $? -ne 0 ]; then
isPortOpen=$PORT_CLOSED
else
isPortOpen=$PORT_OPEN
fi
time=$(($time+$TIME_INTERVAL))
sleep $TIME_INTERVAL
done
if [ $isPortOpen -eq $PORT_OPEN ]; then
echo "Port is open after ${time} seconds."
# Give the server more time to properly start
sleep 10
else
echo "Reached the timeout (${TIMEOUT} seconds). The port ${SERVER}:${PORT} is not available."
exit 1
fi
After running all the test, I just got back:
Reached the timeout (180 seconds). The port localhost:8888 is not available.
I couldn't find out if there were any problems starting the devserver or querying the port.
Does anyone have an idea or solution?
Thanks!
By default, by only accepting localhost|loopback traffic, you're unable to access the server remotely.
Please try adding --address=0.0.0.0:
(link) to your java_dev_appserver.sh command.
Example
Used a variant of Google's HelloWorld sample.
Ran this with mvn appengine:run (to confirm it works and build the WAR).
Then /path/to/bin/java_dev_appserver.sh ./target/hellofreddie-0.1 (to confirm it works with the local development server).
Then used Google's Cloud SDK container image (link), mounted the previously generated WAR directory into it, and ran the server on :9999:
docker run \
--interactive \
--tty \
--publish=9999:9999 \
--volume=${PWD}/target:/target \
google/cloud-sdk \
/usr/lib/google-cloud-sdk/bin/java_dev_appserver.sh \
--address=0.0.0.0 \
--port=9999 \
./target/hellofreddie-0.1
Am able to curl the endpoint:
curl \
--silent \
--location \
--write-out "%{http_code}" \
--output /dev/null \
localhost:9999
returns 200
And, running your scripts adjusted with PORT=9999 returns:
Port is open after 2 seconds.
I'm using Docker steps as below to bring up selenium grid
My query is how to gracefully wait till particular container is downloaded , up and running.
docker run -ti -m 150M --memory-swap 300M --cpu-shares=104 -d -p 4444:4444 --name selenium-hub -e GRID_BROWSER_TIMEOUT=15000 selenium/hub
sleep 10
for i in {1..2}
do
echo "Starting Node: $i"
docker run -ti -m 750M --memory-swap 900M --cpu-shares=460 -d --link selenium-hub:hub -v /dev/shm:/dev/shm selenium/node-chrome
sleep 5
done
Is there better way of avoiding sleep , as sometimes container download takes longer.
After the job is done , I stop and remove all the containers in order to do fresh start for new job.
Thanks & Regards,
Vikram
You can call the selenium service with curl and check the result:
Then in a while check for COUNT value if it is different from zero
COUNT=$(curl -q localhost:4444 | grep 403 | wc -l)
while [ $COUNT -eq 0 ]
do
sleep 1
COUNT=$(curl -q localhost:4444 | grep 403 | wc -l)
done
Regards
I'm using reverse ssh for connecting to remote client , Operator run reverse one time and leave client system
how can i write bash script , when reverse ssh disconnected from server retry to connect to server (ssh)
Use autossh. Autossh "automatically restart[s] SSH sessions and tunnels"
sudo apt-get install autossh
I use autossh to to keep open reverse tunnel that I depend on. It works very well, even with long periods of lost connection.
Here is the script I use to create the tunnel:
#!/bin/bash
AUTOSSH_GATETIME=0
export AUTOSSH_GATETIME
autossh -f -N -R 8022:localhost:22 username#host -o "ServerAliveInterval 45" -o "ServerAliveCountMax 2"
I execute this script at boot with this cronjob:
#reboot /home/scripts/./persistent-tunnel.sh
If you simply want to retry a command until it succeeds, you can use this pattern:
while ! ssh [...]
do
echo "Command failed, retrying..." >&2
done
I have a slightly different method.
My method always tries to reconnect you if you have a dirty disconnection: '~.' or 'Connection closed by remote host.'
But if you disconnect with 'CRTL+D' or with 'exit' it just disconnects and show you some info of the connections.
I have a slightly different method.
My method always tries to reconnect you if you have a dirty disconnection: '~.' or 'Connection closed by remote host.'
But if you disconnect with 'CRTL+D' or with 'exit' it just disconnects and show you some info of the connections.
#/bin/bash
if [ -z "$1" ]
then
echo '''
Please also provide ssh connection details.
'''
exit 1
fi
retries=0
repeat=true
today=$(date)
while "$repeat"
do
((retries+=1)) &&
echo "Try number $retries..." &&
today=$(date) &&
ssh "$#" &&
repeat=false
sleep 5
done
echo """
Disconnected sshx after a successful login.
Total number of tries = $retries
Connected at:
$today
"""
You might want to take a look into ssh options ServerAliveInterval, ServerAliveCountMax and TCPKeepAlive because sometimes your line dies without making this obvious, let me demonstrate:
#!/bin/sh
while true; do
ssh -T user#host \
-o IdentityFile=~/.ssh/tunnel \
-o UserKnownHostsFile=~/.ssh/known_hosts.tunnel \
pkill -f "^sshd:\ user\ \ \ \ $" # needs to be edited for nearly every case
sleep 2
ssh -T -N user#host \
-o IdentityFile=~/.ssh/tunnel \
-o UserKnownHostsFile=~/.ssh/known_hosts.tunnel \
-o Batchmode=yes \
-o ExitOnForwardFailure=yes \
-o ServerAliveCountMax=1 \
-o ServerAliveInterval=60 \
-o LocalForward=127.0.0.1:2501=127.0.0.1:25 \
-o RemoteForward=127.0.0.1:2501=127.0.0.1:25
sleep 60
done
You can use netstat -ntp | grep ":22" or ss -ntp | grep ":22" to see established connections to ssh port, then use grep to filter the ip address you're looking for. If you don't find a connection then reconnect the tunnel.
Use autossh if it works on your version of Linux. It did not on mine as it was an outdated Linux distribution for a custom NAS box.
The alternative is a simple bash script in crontab like this:
maintain_reverse_ssh_tunnel.sh
if ! netstat -planet |grep myserver_ip_or_name |grep ESTABLISHED > /dev/null; then
echo "REVERSE SSH DOWN - Restarting the tunnels"
ssh -fN -R 32999:localhost:22 -R 28080:localhost:80 myusername#myserver_ip_or_name
fi
Replace myusername and myserver_ip_or_name with those of your user and server.
Then add an entry to crontab by typing crontab -e and adding the following line:
1 * * * * /path_to_my_script/maintain_reverse_ssh_tunnel.sh
Make sure to have the execute permissions on the script:
chmod 755 maintain_reverse_ssh_tunnel.sh