For example:
ls -l -d */ | wc -l | awk '{print $1}' | tee /dev/tty | ls -l
This shell command print the result of wc and ls -l with single line, but tee is used.
Is it possible to using one Shell commend line to achieve multiple output without using “&&” “||” “>” “>>” “<” “;” “&”,tee and temp file?
When you want the output of date and ls -rtl | head -1 on one line, you can use
echo "$(date): $(ls -rtl | head -1)"
Yes, you can achieve writing to multiple files with awk which is not in the list of things you appear not to like:
echo hi | awk '{print > "a.txt"; print > "b.txt"}'
Then check a.txt and b.txt.
Related
This is what my loop contains:
cat /$f/stat | awk '{print $1,$3,$4,$7,$17}' /$f/stat
cd $f
sudo ls fd | wc -l
cd ..
At first, it shows the output of:
cat /$f/stat | awk '{print $1,$3,$4,$7,$17}' /$f/stat
And it prints the output of this on a new line:
cd $f
sudo ls fd | wc -l
cd ..
How do I combine these so that it shows them on one line?
At the outset, use shellcheck to validate your script.
Looks like you want awk's output and wc -l's output to be on the same line. Use command substitution for this:
printf '%s %s\n' "$(awk '{print $1,$3,$4,$7,$17}' "$f/stat")" "$(sudo ls "$f/fd" | wc -l)"
no need for cat | awk which is a case of UUOC - awk is reading input from the file passed as an argument; also, it looks like you need "$f/stat" and not "/$f/stat"
enclose variables in double quotes to prevent word splitting and globbing
use full path $f/fd instead of having to do a cd $f and back
Since parsing ls output is considered a bad practice, you could do this instead, on Linux:
printf '%s %s\n' "$(awk '{print $1,$3,$4,$7,$17}' "$f/stat")" "$(sudo find "$f/fd" -maxdepth 1 -print0 | tr -cd '\0' | wc -c)"
find ... -print0 prints NUL terminated list of files
tr -cd '\0' - deletes all characters other than NUL
wc -c - counts the number of NULs, which is the number of file names in find output
How to feed xargs to a piped grep for a piped cat command.
Command 1:
(Generates a grep pattern with unique PIDs for a particular date time, read from runtime.log)
cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='%
The output of above command is (It's custom grep pattern):
2018/09/13 14:50.*PID=13109
2018/09/13 14:50.*PID=14575
2018/09/13 14:50.*PID=15741
Command 2:
(Reads runtime.log and fetch the appropriate lines based on the grep pattern (Ideally the grep pattern should comes from command 1))
cat runtime.log | grep '2018/09/13 14:50.*PID=13109'
The question is How to combine both Command 1 & Command 2
Below combined version of command doesn't gives the expected output (The produced output had lines having the date other than '2018/09/13 14:50')
cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='% | cat runtime.log xargs grep
grep has an option -f. From man grep:
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file contains zero patterns, and therefore matches nothing. (-f is specified by POSIX .)
So you could use
cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='% > a_temp_file
cat runtime.log | grep -f a_temp_file
The shell has a syntax that avoids having to create the temporary file. <(). From man bash:
Process Substitution
Process substitution is supported on systems that support named pipes
(FIFOs) or the /dev/fd method of naming open files. It takes the form
of <(list) or >(list). The process list is run with its input or
output connected to a FIFO or some file in /dev/fd. The name of this
file is passed as an argument to the current command as the result of
the expansion. If the >(list) form is used, writing to the file will
provide input for list. If the <(list) form is used, the file passed
as an argument should be read to obtain the output of list.
So you can combine it to:
cat runtime.log | grep -f <(cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='%)
i want to svn blame lines of code which include "todo | fixme"
i have the general flow of the script but struggle to combine it into one
finding the lines with "todo"
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" .
blame the line of code
svn blame ${file} | cat -n |grep ${linenumber}
i could get $file and $linenumber from the first command with awk, but i dont know how to pipe the values i extract with awk into the second command.
i am missing the glue to combine these commands into one "script" (- :
You can build the command with awk and then pipe it to bash:
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" . |\
awk -F: '{printf "svn blame \"%s\" | cat -n | grep \"%s\"\n", $1, $2}'
That prints one command per input line with the following format:
svn blame "${file}" | cat -n | grep "${linenumber}"
The varibales are replaces. When you execute the command as above they are only printed to the shell, that you can comfirm if everything is right. If yes add a last pipe to the in of the command that the ouput is redirected to bash. The complete command would look like this:
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" . |\
awk -F: '{printf "svn blame \"%s\" | cat -n | grep \"%s\"\n", $1, $2}' | bash
A small notice: I think you want to print the line number extracterd in the first command, aren't you? But grep ${linenumber} just gives the line containing the string ${linenumber}. To print only the linenumber use that command: sed -n "2p" to print line number 2 for example. The complete command would then look like this:
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" . |\
awk -F: '{printf "svn blame \"%s\" | cat -n | sed -n \"%sp\"\n", $1, $2}' | bash
I have the following three constructs in a bash script:
NUMOFLINES=$(wc -l $JAVA_TAGS_FILE)
echo $NUMOFLINES" lines"
echo $(wc -l $JAVA_TAGS_FILE)" lines"
echo "$(wc -l $JAVA_TAGS_FILE) lines"
And they both produce identical output when the script is run:
121711 /home/slash/.java_base.tag lines
121711 /home/slash/.java_base.tag lines
121711 /home/slash/.java_base.tag lines
I.e. the name of the file is also echoed (which I don't want to). Why do these scriplets fail and how should I output a clean:
121711 lines
?
An Example Using Your Own Data
You can avoid having your filename embedded in the NUMOFLINES variable by using redirection from JAVA_TAGS_FILE, rather than passing the filename as an argument to wc. For example:
NUMOFLINES=$(wc -l < "$JAVA_TAGS_FILE")
Explanation: Use Pipes or Redirection to Avoid Filenames in Output
The wc utility will not print the name of the file in its output if input is taken from a pipe or redirection operator. Consider these various examples:
# wc shows filename when the file is an argument
$ wc -l /etc/passwd
41 /etc/passwd
# filename is ignored when piped in on standard input
$ cat /etc/passwd | wc -l
41
# unusual redirection, but wc still ignores the filename
$ < /etc/passwd wc -l
41
# typical redirection, taking standard input from a file
$ wc -l < /etc/passwd
41
As you can see, the only time wc will print the filename is when its passed as an argument, rather than as data on standard input. In some cases, you may want the filename to be printed, so it's useful to understand when it will be displayed.
wc can't get the filename if you don't give it one.
wc -l < "$JAVA_TAGS_FILE"
You can also use awk:
awk 'END {print NR,"lines"}' filename
Or
awk 'END {print NR}' filename
(apply on Mac, and probably other Unixes)
Actually there is a problem with the wc approach: it does not count the last line if it does not terminate with the end of line symbol.
Use this instead
nbLines=$(cat -n file.txt | tail -n 1 | cut -f1 | xargs)
or even better (thanks gniourf_gniourf):
nblines=$(grep -c '' file.txt)
Note: The awk approach by chilicuil also works.
It's a very simple:
NUMOFLINES=$(cat $JAVA_TAGS_FILE | wc -l )
or
NUMOFLINES=$(wc -l $JAVA_TAGS_FILE | awk '{print $1}')
I normally use the 'back tick' feature of bash
export NUM_LINES=`wc -l filename`
Note the 'tick' is the 'back tick' e.g. ` not the normal single quote
Is there any bash command that will let you get the nth line of STDOUT?
That is to say, something that would take this
$ ls -l
-rw-r--r--# 1 root wheel my.txt
-rw-r--r--# 1 root wheel files.txt
-rw-r--r--# 1 root wheel here.txt
and do something like
$ ls -l | magic-command 2
-rw-r--r--# 1 root wheel files.txt
I realize this would be bad practice when writing scripts meant to be reused, BUT when working with the shell day to day it'd be useful to me to be able to filter my STDOUT in such a way.
I also realize this would be semi-trivial command to write (buffer STDOUT, return a specific line), but I want to know if there's some standard shell command to do this that would be available without me dropping a script into place.
Using sed, just for variety:
ls -l | sed -n 2p
Using this alternative, which looks more efficient since it stops reading the input when the required line is printed, may generate a SIGPIPE in the feeding process, which may in turn generate an unwanted error message:
ls -l | sed -n -e '2{p;q}'
I've seen that often enough that I usually use the first (which is easier to type, anyway), though ls is not a command that complains when it gets SIGPIPE.
For a range of lines:
ls -l | sed -n 2,4p
For several ranges of lines:
ls -l | sed -n -e 2,4p -e 20,30p
ls -l | sed -n -e '2,4p;20,30p'
ls -l | head -2 | tail -1
Alternative to the nice head / tail way:
ls -al | awk 'NR==2'
or
ls -al | sed -n '2p'
From sed1line:
# print line number 52
sed -n '52p' # method 1
sed '52!d' # method 2
sed '52q;d' # method 3, efficient on large files
From awk1line:
# print line number 52
awk 'NR==52'
awk 'NR==52 {print;exit}' # more efficient on large files
For the sake of completeness ;-)
shorter code
find / | awk NR==3
shorter life
find / | awk 'NR==3 {print $0; exit}'
Try this sed version:
ls -l | sed '2 ! d'
It says "delete all the lines that aren't the second one".
You can use awk:
ls -l | awk 'NR==2'
Update
The above code will not get what we want because of off-by-one error: the ls -l command's first line is the total line. For that, the following revised code will work:
ls -l | awk 'NR==3'
Another poster suggested
ls -l | head -2 | tail -1
but if you pipe head into tail, it looks like everything up to line N is processed twice.
Piping tail into head
ls -l | tail -n +2 | head -n1
would be more efficient?
Is Perl easily available to you?
$ perl -n -e 'if ($. == 7) { print; exit(0); }'
Obviously substitute whatever number you want for 7.
Yes, the most efficient way (as already pointed out by Jonathan Leffler) is to use sed with print & quit:
set -o pipefail # cf. help set
time -p ls -l | sed -n -e '2{p;q;}' # only print the second line & quit (on Mac OS X)
echo "$?: ${PIPESTATUS[*]}" # cf. man bash | less -p 'PIPESTATUS'
Hmm
sed did not work in my case.
I propose:
for "odd" lines 1,3,5,7... ls |awk '0 == (NR+1) % 2'
for "even" lines 2,4,6,8 ls |awk '0 == (NR) % 2'
For more completeness..
ls -l | (for ((x=0;x<2;x++)) ; do read ; done ; head -n1)
Throw away lines until you get to the second, then print out the first line after that. So, it prints the 3rd line.
If it's just the second line..
ls -l | (read; head -n1)
Put as many 'read's as necessary.