The following command processes the output of the pipe twice by using tee:
echo -e "ALPHA\nBRAVO" | tee >(head -n 1) >(tail) >/dev/null
As expected it outputs:
ALPHA
ALPHA
BRAVO
When trying to call it with watch like this:
watch 'echo -e "ALPHA\nBRAVO" | tee >(head -n 1) >(tail) >/dev/null'
It returns:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `echo -e "ALPHA\nBRAVO" | tee >(head -n 1) >(tail) >/dev/null'
How should I escape my command to use it with watch?
Process substitutions are an extension, not all sh implementations support them. You can use redirections to circumvent this restriction though. Like
watch '{ { printf '\''ALPHA\nBRAVO\n'\'' |
tee /proc/self/fd/3 |
head -n 1 >&4
} 3>&1 | tail >&4
} 4>&1'
Just note that this is no more portable than doing watch 'bash -c ...'.
I have a bash function
agg_generror () {
echo $1
find ${folder} -name "${prefix}_*_${1}_${suffix}.count" | xargs -I % sh -c 'cat %; echo "";' | awk 'BEGIN{e=0;t=0} {e+=$1;t+=$2} END{print e/t}' > generror_${1}
}
which if I call directly
agg_generror 17.5
works and doesn't complain.
But if I do
echo 17.5 | xargs -I % sh -c 'agg_generror %'
It fails with
17.5
awk: fatal: division by zero attempted
Why may the behaviour different in the two cases?
while read; do agg_generror $REPLY; done < input.txt
I have a command line tool which receives two arguments:
TOOL arg1 -o arg2
I would like to invoke it with the same argument provided it for arg1 and arg2, and to make that easy for me, i thought i would do:
each <arg1_value> | TOOL $1 -o $1
but that doesn't work, $1 is not replaced, but is added once to the end of the commandline.
An explicit example, performing:
cp fileA fileA
returns an error fileA and fileA are identical (not copied)
While performing:
echo fileA | cp $1 $1
returns the following error:
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory
any ideas?
If you want to use xargs, the [-I] option may help:
-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separa‐
tor is the newline character. Implies -x and -L 1.
Here is a simple example:
mkdir test && cd test && touch tmp
ls | xargs -I '{}' cp '{}' '{}'
Returns an Error cp: tmp and tmp are the same file
The xargs utility will duplicate its input stream to replace all placeholders in its argument if you use the -I flag:
$ echo hello | xargs -I XXX echo XXX XXX XXX
hello hello hello
The placeholder XXX (may be any string) is replaced with the entire line of input from the input stream to xargs, so if we give it two lines:
$ printf "hello\nworld\n" | xargs -I XXX echo XXX XXX XXX
hello hello hello
world world world
You may use this with your tool:
$ generate_args | xargs -I XXX TOOL XXX -o XXX
Where generate_args is a script, command or shell function that generates arguments for your tool.
The reason
each <arg1_value> | TOOL $1 -o $1
did not work, apart from each not being a command that I recognise, is that $1 expands to the first positional parameter of the current shell or function.
The following would have worked:
set - "arg1_value"
TOOL "$1" -o "$1"
because that sets the value of $1 before calling you tool.
You can re-run a shell to perform variable expansion, with sh -c. The -c takes an argument which is command to run in a shell, performing expansion. Next arguments of sh will be interpreted as $0, $1, and so on, to use in the -c. For example:
sh -c 'echo $1, i repeat: $1' foo bar baz will print execute echo $1, i repeat: $1 with $1 set to bar ($0 is set to foo and $2 to baz), finally printing bar, i repeat: bar
The $1,$2...$N are only visible to bash script to interpret arguments to those scripts and won't work the way you want them to. Piping redirects stdout to stdin and is not what you are looking for either.
If you just want a one-liner, use something like
ARG1=hello && tool $ARG1 $ARG1
Using GNU parallel to use STDIN four times, to print a multiplication table:
seq 5 | parallel 'echo {} \* {} = $(( {} * {} ))'
Output:
1 * 1 = 1
2 * 2 = 4
3 * 3 = 9
4 * 4 = 16
5 * 5 = 25
One could encapsulate the tool using awk:
$ echo arg1 arg2 | awk '{ system("echo TOOL " $1 " -o " $2) }'
TOOL arg1 -o arg2
Remove the echo within the system() call and TOOL should be executed in accordance with requirements:
echo arg1 arg2 | awk '{ system("TOOL " $1 " -o " $2) }'
Double up the data from a pipe, and feed it to a command two at a time, using sed and xargs:
seq 5 | sed p | xargs -L 2 echo
Output:
1 1
2 2
3 3
4 4
5 5
I haven't worked much with solaris, but I'm supposed to be writing a script that searches for a line in a file, comments it out, and writes the correct line below it.
for i in `cat solarishosts`
do
#print hostname
echo ${i}
#get the line number of the expression after the /; save its value to linenum
linenum="$(ssh -o ConnectTimeout=1 -o ConnectionAttempts=1 ${i} "awk '/%sugrp ALL=\(user\) lines: /usr/bin/su -, /usr/bin/su - user/a{ print NR; exit }' /usr/local/etc/sudoers")"
#overwrite the line # linenum (overwriting just a to add a comment)
ssh -o ConnectTimeout=1 -o ConnectionAttempts=1 ${i} "sed -n "${linenum}"p <<< "#%sugrp ALL=\(user\) lines: /usr/bin/su -, /usr/bin/su - user""
#use the linenum var to make a newlinenum var , this one being one line down from where the commented text was written
newlinenum=linenum+1
#write the line in quotes # the newlinenum position
ssh -o ConnectTimeout=1 -o ConnectionAttempts=1 ${i} "sed -n "${newlinenum}"p <<< "%sugrp ALL=\(ALL\) ALL""
done
I'm getting weird errors :
awk: syntax error near line 1
awk: bailing out near line 1
bash: -c: line 0: syntax error near unexpected token `newline'
bash: -c: line 0: `sed -n p <<< #%sugrp ALL=(user) PASSWD: /usr/bin/su -, /usr/bin/su - user'
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `sed -n linenum+1p <<< %sugrp ALL=(ALL) ALL'
It looks like there's an error with my awk syntax ... but it isn't on line 1? And I'm not sure what the error is
I don't have a newline character anywhere in my first sed line?
In my code I escaped the "(' it's complaining about
That's pretty messy. You don't need to ssh into the box 3 times. Your quoting is a big problem. And you never actually write the changes back to the file.
Try this: build up the remote command and call ssh once:
line='%sugrp ALL=(user) lines: /usr/bin/su -, /usr/bin/su - user'
newline='%sugrp ALL=(ALL) ALL'
file=/usr/local/etc/sudoers
awkcmd='$0 == line {print "#" $0; print new}'
cmd=$(
printf "awk -v line='%s' -v new='%s' '%s' %s > %s.new && mv %s %s.bak && mv %s.new %s" \
"$line" \
"$newline" \
"$awkcmd" \
"$file" "$file" "$file" "$file" "$file" "$file"
)
while read -r host; do
echo "$host"
# perform the remote command
ssh -o ConnectTimeout=1 -o ConnectionAttempts=1 "$host" sh -c "$cmd"
done < solarishosts
I use single quotes as much as possible to reduce the need for backslashes in the constant strings, and all variables are quoted when used.
I have a line working from this thread that tails a file until a matching pattern is found. It works well, but I can't find a way to suppress the output that occurs afterwards.
The line is:
sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$ ;}'
piping to /dev/null doesn't work as I don't get any output at all from the tail command that way. Also, I'm on OSX and various other sed and awk suggestions don't work due to the syntax.
It always finishes with the below, instead of nothing:
sh: line 10: 14285 Terminated: 15 sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$ ;}'
I'd also like not to display the matched text (EOF in the above example).
Any suggestions welcomed.
for a file (log for example)
sed -u "/pattern/ q" YourFile
for a pipe
ls -l | sed -u "/pattern/ q"
the -u of sed tell it to work as a stream input
It's the shell's job monitoring popping the message.
nomonitor() {
set +m
"$#"
set -m
}
nomonitor sh -c 'tail -n +0 -f $logfile | { sed "/EOF/ q" && kill $$; }'
You can actually discard the stderr like this:
sh -c 'tail -n +0 -f $logfile | { sed "/EOF/q" && p=$$ && kill $((p+1)) ; }'