I have a system that handles incoming emails to send them to a blackbox application at my job. The high level script is maintained by inittab to always run and runs a child script to do the actual work with this command:
$SCRIPT | nc -l -p $PORT
The script itself reads from a named pipe, does a bit of parsing and processing of the data before calling echo to shuffle the data back through netcat to the process connected on $PORT.
What I need is some method to handle incoming data from the far end of my pipe. When I make a request within the application to close the connection it sends back a string (I can define it to whatever I want) and waits for my script to close the pipe. I currently am struggling to understand how I can add in the functionality to read incoming data from the other end; verify it is the command to close the pipe, and then exit the script.
My script (in a nutshell) looks like this:
while true ; do
email_input="`cat "$pipe"`"
if [[ $email_input =~ .*escape_queue.* ]] ; then
break;
fi
echo "`parse`"
done
I'm open to the possibility of having to alter the program flow, I just can't wrap my head around how I would be able to read the data incoming asynchronously since the script blocks on cat $pipe until a new email is received to process.
If its not clear, I'm at a novice level with bash scripting and am always open to suggestions for improvement.
UPDATE
I've changed my script call to
$SCRIPT | nc -l -p $PORT > $nc_data
and within the script itself
netcat_response="`cat "$nc_data"`";
if [[ "$netcat_response" =~ "exit" ]] ; then
cat /dev/null > $nc_data
break;
fi
At this point the script terminates once a new message is piped into the fifo. This means that I will always lose 1 message as it gets read by the script and then the script terminates. The script still blocks on the cat until something is read. Worst case scenario this will have to do.
You can have nc quit after a certain time from the EOF of stdin.
$SCRIPT | nc -l -q 5 -p $PORT > $nc_data
-q being the option to quit after a certain amount of seconds.
Related
I need to sendo a lot of messages via netcat or something similar. The problem is that when I run echo "something" | netcat ip port the connection continues opened after I received the response. Actually the connection continues opened waiting for a new input. However, what I need is that the connection closed after I receive the response. Look, my script is basically this:
#!/bin/bash
i=1
while [ $i -ne 10000 ];do
sed -n $[i]p wordlist | netcat localhost 30002 >> result
i=$[$i+1]
done
If I can close the connection after print the response in result, everything will work fine. I know that there is an option -w "x" that closes the connection after "x" seconds, but the minimum value for "x" is 1 and 1 is bigger than I can wait, I need close the connection as soon as possible.
Unfortunately, the -q flag didn't work for me.
I'm using "OpenBSD netcat (Debian patchlevel 1.187-1ubuntu0.1)" and, even though the -q flag shows up in the manual, it didn't work as mentioned in cnicutar's answer.
Therefore, my workaround was:
#!/bin/sh
# POSIX COMPLIANT
HOST="localhost"
PORT=30002
scan () {
# Ensuring there is no file named msg
rm msg
# While msg file doesn't exist or is empty, do
while [ ! -s msg ]; do
# Remove instruction from within the loop
rm msg
# Append the received messages to msg file, and put the process in the background
echo "$HOST $PORT" | xargs nc >> msg &
# If the file exists and is not empty, return, we received the message
[ -s msg ] && return;
# A small timeout.. doing some tests I noticed that a timeout of zero sometimes didn't work to catch the message
# Maybe nc needs a small time to receive everything. You might want to test and increase or decrease this timeout if needed.
sleep 0.1
# This script will be spawning a lot of nc process, to kill it before the loop runs again
pkill -x nc
done
} 2> /dev/null
scan
# The function returned, so cat the file
cat msg
# make sure nc is killed
pkill -x nc > /dev/null 2>&1
rm msg
What you're looking for is the -q switch. If you specify:
netcat -q 0 localhost 30002
netcat will exit immediately.
I need a way to fire a netcat listener from a shell script and if a connection received I need to run a command on the same local listener machine and without interrupting the netcat process / connection
it's like the -e option but I need to run a command locally while keeping the netcat connection running
I don't really know if it can be done I mean after the shell process forked the netcat child can it interact with nc's output for example and run other command before netcat exit?
Edit: I figured it's even easier to do it on the client C code side by checking the return value of an initial send() message to determine if the client connected successfully if we got the sent message length
sret = send(sock, message, strlen(message), 0);
if (sret == strlen(message)) // We're Connected
do something
Thanks
This will check if the initial nc process has started listening, and it will echo every line of input it receives and will then send back a Received response:
rm -f input.txt
touch input.txt
tail -f input.txt | nc -l 5555 > output.txt &
if ! ps -p $! >/dev/null; then
echo "Netcat didn't start. Exiting..."
exit 1
fi
tail -f output.txt | while read -r LINE; do
echo "Received input: $LINE"
echo "Received" >> input.txt
done
See if you can adapt this to meet your needs.
I've created a named pipe for some other process to write to and want to check that the other process started correctly, but don't know its PID. The context is running a command in screen, making sure the command started correctly. I was hoping this might work:
mkfifo /tmp/foo
echo hello > /tmp/foo &
lsof /tmp/foo
Sadly, lsof does not report echo. inotifywait might be another option, but isn't always installed and I really want to poll just once, rather than block until some event.
Is there any way to check if a named pipe is open for writing? Even open in general?
UPDATE:
Once both ends are connected lsof seems to work. This actually solves my problem, but for the sake of the question I'd be interested to know if it's possible to detect the initial redirection to the named pipe without a reader.
> mkfifo /tmp/foo
> yes > /tmp/foo &
> lsof /tmp/foo
> cat /tmp/foo > /dev/null &
> lsof /tmp/foo
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
yes 16915 user 1w FIFO 8,18 0t0 16660270 /tmp/foo
cat 16950 user 3r FIFO 8,18 0t0 16660270 /tmp/foo
Update 2: After playing with inotify-tools, there doesn't seem to be a way to get a notification that a named pipe has been opened for writing and is blocking. This is probably why lsof doesn't show the pipe until it has a reader and a writer.
Update: After researching named pipes, I don't believe that there is any method that will work with named pipes by themselves.
Reasoning:
there is no way to limit the number of writers to a named pipe (without resorting to locking)
all writers block if there is no reader
no writers block if there is a reader (presumably as long as the kernel buffers aren't full)
You could try writing nothing to the pipe with a short timeout. If the timeout expires, then the write blocked indicating that someone has already opened the pipe for writing.
Note: As pointed out in the comments, if a reader exists and presumably is fast enough, our test write will not block and the test essentially fails. Comment out the cat line below to test this.
#!/bin/bash
is_named_pipe_already_opened_for_writing() {
local named_pipe="$1"
# Make sure it's a named pipe
if ! [ -p "$named_pipe" ]; then
return 1
fi
# Try to write zero bytes in the background
echo -n > "$named_pipe" &
pid=$!
# Wait a short amount of time
sleep 0.1
# Kill the background process. If kill succeeds, then
# the write was blocked indicating that someone
# else is already writing to the named pipe.
kill $pid 2>/dev/null
}
PIPE=/tmp/foo
# Ignore any bash messages from killing below
trap : TERM
mkfifo $PIPE
# a writer
yes > $PIPE &
# a reader
cat $PIPE >/dev/null &
if is_named_pipe_already_opened_for_writing "$PIPE"; then
echo "$PIPE is already being written to by another process"
else
echo "$PIPE is NOT being written to by another process"
fi
jobs -pr | kill 2>/dev/null
rm -f $PIPE
you need two pipes one for each directions:
one is use to wait for the ready for new data signal, another just for the data:
in my case process to files, line by line:
mkfifo r w;
cat file1 | while read l; do echo "$l" >w; read <r; done &
cat file2 | while read ln; do if read l <w; then echo "$ln"; echo "$l"; fi; echo 1>r; done
I'm running a bash script that goes through the list of my remote server IPs, connects via netcat (telnet) for each line, and runs a few commands.
The problem is I can't seem to figure out how to terminate netcat so the script can loop to the next IP in the list.
Here's the relevant bit:
#!/bin/bash
while ISF= read -r line;do
(
sleep 3
printf 'command1'
sleep 3
printf 'command2'
sleep 3
) | nc $line
done < ~/servers.txt
The remote servers don't send an EOF, so is there something I can echo or printf at netcat to terminate netcat so the script can loop through again? I would really rather not do a -w flag for a timeout, because I have quite a few servers I need to do this on, and a timeout would make it take much longer.
Specify a timeout after which nc will exit if it receives no further input, either from the remote end or via standard input.
... | nc "$line" -w 10 # Choose a value for -w as appropriate
Depends on your version of netcat, but -c should do what your looking for. From the usage statement of gnu netcat (which is likely what you're running on Ubuntu):
-c, --close close connection on EOF from stdin
Abstract: How to run an interactive task in background?
Details: I am trying to run this simple script under ash shell (Busybox) as a background task.
myscript.sh&
However the script stops immediately...
[1]+ Stopped (tty input) myscript.sh
The myscript.sh contents... (only the relvant part, other then that I trap SIGINT, SIGHUP etc)
#!/bin/sh
catpid=0
START_COPY()
{
cat /dev/charfile > /path/outfile &
catpid = $!
}
STOP_COPY()
{
kill catpid
}
netcat SOME_IP PORT | while read EVENT
do
case $EVENT in
start) START_COPY;;
stop) STOP_COPY;;
esac
done
From simple command line tests I found that bot cat and netcat try to read from tty.
Note that this netcat version does not have -e to supress tty.
Now what can be done to avoid myscript becoming stopped?
Things I have tried so for without any success:
1) netcat/cat ... < /dev/tty (or the output of tty)
2) Running the block containing cat and netcat in a subshell using (). This may work but then how to grab PID of cat?
Over to you experts...
The problem still exists.
A simple test for you all to try:
1) In one terminal run netcat -l -p 11111 (without &)
2) In another terminal run netcat localhost 11111 & (This should stop after a while with message Stopped (TTY input) )
How to avoid this?
you probably want netcat's "-d" option, which tells it not to read from STDIN.
I can confirm that -d will help netcat run in the background.
I was seeing the same issue with:
nc -ulk 60001 | nc -lk 60002 &
Every time I queried the jobs, the pipe input would stop.
Changing the command to the following fixed it:
nc -ulkd 60001 | nc -lk 60002 &
Are you sure you've given your script as is or did you just type in a rough facsimile meant to illustrate the general idea? The script in your question has many errors which should prevent it from ever running correctly, which makes me wonder.
The spaces around the = in catpid=$! make the line not a valid variable assignment. If that was in your original script I am surprised you were not getting any errors.
The kill catpid line should fail because the literal word catpid is not a valid job id. You probably want kill "$catpid".
As for your actual question:
cat should be reading from /dev/charfile and not from stdin or anywhere else. Are you sure it was attempting to read tty input?
Have you tried redirecting netcat's input like netcat < /dev/null if you don't need netcat to read anything?
I have to use a netcat that doesn't have the -d option.
"echo -n | netcat ... &" seems to be an effective workaround: i.e. close the standard input to netcat immediately if you don't need to use it.
As it was not yet really answered, if using Busybox and -d option is not available, the following command will keep netcat "alive" when sent to background:
tail -f /dev/null | netcat ...
netcat < /dev/null and echo -n | netcat did not work for me.
Combining screen and disown-ing process work for me, as '-d' option is not a valid anymore for netcat. Tried redirecting like nc </dev/null but session ends prematurely (as I need -q 1 to make sure nc process stop as file transfer finished)
Setup Receiver side first,
on Receiver side, screen keep stdin for netcat so it won't terminated
EDIT: I was wrong, you need to enter command INSIDE screen. You'll end with no file saved, or weird binary thing flow in your terminal while attach to screen, if you redirecting nc inline of screen command. (Example, this is THE WRONG WAY: screen nc -l -p <listen port> -q 1 > /path/to/yourfile.bin)
Open screen , then press return/Enter on welcome message. new blank shell will appear (you're inside screen now)
type command: nc -l -p 1234 > /path/to/yourfile.bin
then, press CTRL + a , then press d to detach screen.
on Sender sides, disown process, quit 1s after reaching EOF
cat /path/to/yourfile.bin | nc -q1 100.10.10.10 1234 & disown