The following command processes the output of the pipe twice by using tee:
echo -e "ALPHA\nBRAVO" | tee >(head -n 1) >(tail) >/dev/null
As expected it outputs:
ALPHA
ALPHA
BRAVO
When trying to call it with watch like this:
watch 'echo -e "ALPHA\nBRAVO" | tee >(head -n 1) >(tail) >/dev/null'
It returns:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `echo -e "ALPHA\nBRAVO" | tee >(head -n 1) >(tail) >/dev/null'
How should I escape my command to use it with watch?
Process substitutions are an extension, not all sh implementations support them. You can use redirections to circumvent this restriction though. Like
watch '{ { printf '\''ALPHA\nBRAVO\n'\'' |
tee /proc/self/fd/3 |
head -n 1 >&4
} 3>&1 | tail >&4
} 4>&1'
Just note that this is no more portable than doing watch 'bash -c ...'.
Related
I try to run the following command (ref.) using nohup, which basically separates stdout and stderr into two processes.
{ foo 2>&1 1>&3 3>&- | sed -u 's/^/err: /'; } 3>&1 1>&2 | sed -u 's/^/out: /'
The foo script is like below.
#!/bin/bash
while true; do
echo a
echo b >&2
sleep 1
done
This is the test result.
$ nohup { foo 2>&1 1>&3 3>&- | sed -u 's/^/err: /'; } 3>&1 1>&2 | sed -u 's/^/out: /' >/dev/null 2>&1 &
-bash: syntax error near unexpected token `}'
That's syntatically impossible. But you can wrap your {} in a sh -c cmd:
nohup sh -c 'foo 2>&1 1>&3 3>&- | sed -u "s/^/err: /"'
Notice I change the single quote for sed to double quote.
watch -n 1 "paste <(ssh ai02 'nvidia-smi pmon -s um -c 1') <(ssh ai03 'nvidia-smi pmon -s um -c 1' )"
The above command is used to horizontally stack two server GPU stats together. It works without the watch command but get the following error
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `paste <(ssh ai02 'nvidia-smi pmon -s um -c 1') <(ssh ai03 'nvidia-smi pmon -s um -c 1' )'
You didn't provide a reproducible example, but I think I managed to make one for testing:
watch -n1 "paste <(seq -w 1000 | shuf -n '10' ) <(seq -w 1000 | shuf -n '10')"
output a similar error:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `paste <(seq -w 1000 | shuf -n '10' ) <(seq -w 1000 | shuf -n '1
0')'
To solve this problem in a simpler way, we can change sh -c for bash -c:
watch -n1 -x bash -c 'paste <(seq -w 1000 | shuf -n "10" ) <(seq -w 1000 | shuf -n "10")'
From the watch manual:
-x, --exec
Pass command to exec(2) instead of sh -c which reduces the need to
use extra quoting to get the desired effect.
If you need maintain the apostrophes from the original commandline, you can escape then too:
watch -e -n1 -x bash -c 'paste <(seq -w 1000 | shuf -n '\''10'\'' ) <(seq -w 1000 | shuf -n '\''10'\'')'
I have an executable that accepts queries from stdin and responds to them, reading until EOF. Additionally I have an input file and a special command, let's call those EXEC, FILE and CMD respectively.
What I need to do is:
Pass FILE to EXEC as input.
Disregard all the output corresponding to commands read from FILE (/dev/null/).
Pass CMD as the last command.
Fetch output for the last command and save it in a variable.
EXEC's output can be multiline for each query.
I know how to pass FILE + CMD into the EXEC:
echo ${CMD} | cat ${FILE} - | ${EXEC}
but I have no idea how to fetch only output resulting from CMD.
Is there a magical one-liner that does this?
After looking around I've found the following partial solution:
mkfifo mypipe
(tail -f mypipe) | ${EXEC} &
cat ${FILE} | while read line; do
echo ${line} > mypipe
done
echo ${CMD} > mypipe
This allows me to redirect my input, but now the output gets printed to screen. I want to ignore all the output produced by EXEC in the while loop and get only what it prints for the last line.
I tried what first came into my mind, which is:
(tail -f mypipe) | ${EXEC} > somefile &
But it didn't work, the file was empty.
This is race-prone -- I'd suggest putting in a delay after the kill, or using an explicit sigil to determine when it's been received. That said:
#!/usr/bin/env bash
# route FD 4 to your output routine
exec 4> >(
output=; trap 'output=1' USR1
while IFS= read -r line; do
[[ $output ]] && printf '%s\n' "$line"
done
); out_pid=$!
# Capture the PID for the process substitution above; note that this requires a very
# new version of bash (4.4?)
[[ $out_pid ]] || { echo "ERROR: Your bash version is too old" >&2; exit 1; }
# Run your program in another process substitution, and close the parent's handle on FD 4
exec 3> >("$EXEC" >&4) 4>&-
# cat your file to FD 3...
cat "$file" >&3
# UGLY HACK: Wait to let your program finish flushing output from those commands
sleep 0.1
# notify the subshell writing output to disk that the ignored input is done...
kill -USR1 "$out_pid"
# UGLY HACK: Wait to let the subprocess actually receive the signal and set output=1
sleep 0.1
# ...and then write the command for which you actually want content logged.
echo "command" >&3
In validating this answer, I'm doing the following:
EXEC=stub_function
stub_function() {
local count line
count=0
while IFS= read -r line; do
(( ++count ))
printf '%s: %s\n' "$count" "$line"
done
}
cat >file <<EOF
do-not-log-my-output-1
do-not-log-my-output-2
do-not-log-my-output-3
EOF
file=file
export -f stub_function
export file EXEC
Output is only:
4: command
You could pipe it into a sed:
var=$(YOUR COMMAND | sed '$!d')
This will put only the last line into the variable
I think, that your proram EXEC does something special (open connection or remember state). When that is not the case, you can use
${EXEC} < ${FILE} > /dev/null
myvar=$(echo ${CMD} | ${EXEC})
Or with normal commands:
# Do not use (printf "==%s==\n" 1 2 3 ; printf "oo%soo\n" 4 5 6) | cat
printf "==%s==\n" 1 2 3 | cat > /dev/null
myvar=$(printf "oo%soo\n" 4 5 6 | cat)
When you need to give all input to one process, perhaps you can think of a marker that you can filter on:
(printf "==%s==\n" 1 2 3 ; printf "%s\n" "marker"; printf "oo%soo\n" 4 5 6) | cat | sed '1,/marker/ d'
You should examine your EXEC what could be used. When it is running SQL, you might use something like
(cat ${FILE}; echo 'select "DamonMarker" from dual;' ; echo ${CMD} ) |
${EXEC} | sed '1,/DamonMarker/ d'
and write this in a var with
myvar=$( (cat ${FILE}; echo 'select "DamonMarker" from dual;' ; echo ${CMD} ) |
${EXEC} | sed '1,/DamonMarker/ d' )
I have a line working from this thread that tails a file until a matching pattern is found. It works well, but I can't find a way to suppress the output that occurs afterwards.
The line is:
sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$ ;}'
piping to /dev/null doesn't work as I don't get any output at all from the tail command that way. Also, I'm on OSX and various other sed and awk suggestions don't work due to the syntax.
It always finishes with the below, instead of nothing:
sh: line 10: 14285 Terminated: 15 sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$ ;}'
I'd also like not to display the matched text (EOF in the above example).
Any suggestions welcomed.
for a file (log for example)
sed -u "/pattern/ q" YourFile
for a pipe
ls -l | sed -u "/pattern/ q"
the -u of sed tell it to work as a stream input
It's the shell's job monitoring popping the message.
nomonitor() {
set +m
"$#"
set -m
}
nomonitor sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$; }'
You can actually discard the stderr like this:
sh -c 'tail -n +0 -f $logfile | { sed "/EOF/q" && p=$$ && kill $((p+1)) ; }'
i need to redirect stout and stderr in bash each to separate file.
well i completed this command:
((/usr/bin/java -jar /opt/SEOC2/seoc2.jar 2>&1 1>&3 | tee --append /opt/SEOC2/log/err.log) 3>&1 1>&2 | tee --append /opt/SEOC2/log/app.log) >> /opt/SEOC2/log/combined.log 2>&1 &
which works fine running from a command line.
trying to put the very same command into bash script
...
12 cmd="(($run -jar $cmd 2>&1 1>&3 | tee --append $err) 3>&1 1>&2 | tee --append $log) >> $combined 2>&1"
...
30 echo -e "Starting servis..."
31 $cmd &
32 pid=`ps -eo pid,args | grep seoc2.jar | grep -v grep | cut -c1-6`
33 if [ ! -z $pid ]; then
...
leads to error like this:
root#operator:/opt/SEOC2# seoc2 start
Starting servis...
/usr/local/bin/seoc2: line 31: ((/usr/bin/java: dir or file doesn't exist
tried to cover this command by $( ), ` ` etc but with no effect at all :(
any suggestion or advice would be very appreciated, playing around for hours already :/
thanx a lot
Rene
If you store the whole command line in a variable you have to use eval to execute it:
cmd="(($run -jar $cmd 2>&1 1>&3 | tee --append $err) 3>&1 1>&2 | tee --append $log) >> $combined 2>&1"
...
eval $cmd &