i want to svn blame lines of code which include "todo | fixme"
i have the general flow of the script but struggle to combine it into one
finding the lines with "todo"
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" .
blame the line of code
svn blame ${file} | cat -n |grep ${linenumber}
i could get $file and $linenumber from the first command with awk, but i dont know how to pipe the values i extract with awk into the second command.
i am missing the glue to combine these commands into one "script" (- :
You can build the command with awk and then pipe it to bash:
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" . |\
awk -F: '{printf "svn blame \"%s\" | cat -n | grep \"%s\"\n", $1, $2}'
That prints one command per input line with the following format:
svn blame "${file}" | cat -n | grep "${linenumber}"
The varibales are replaces. When you execute the command as above they are only printed to the shell, that you can comfirm if everything is right. If yes add a last pipe to the in of the command that the ouput is redirected to bash. The complete command would look like this:
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" . |\
awk -F: '{printf "svn blame \"%s\" | cat -n | grep \"%s\"\n", $1, $2}' | bash
A small notice: I think you want to print the line number extracterd in the first command, aren't you? But grep ${linenumber} just gives the line containing the string ${linenumber}. To print only the linenumber use that command: sed -n "2p" to print line number 2 for example. The complete command would then look like this:
grep --color -Ern --include=*.{php,html,phtml} --exclude-dir=vendor "todo|TODO|FIXME" . |\
awk -F: '{printf "svn blame \"%s\" | cat -n | sed -n \"%sp\"\n", $1, $2}' | bash
Related
For example:
ls -l -d */ | wc -l | awk '{print $1}' | tee /dev/tty | ls -l
This shell command print the result of wc and ls -l with single line, but tee is used.
Is it possible to using one Shell commend line to achieve multiple output without using “&&” “||” “>” “>>” “<” “;” “&”,tee and temp file?
When you want the output of date and ls -rtl | head -1 on one line, you can use
echo "$(date): $(ls -rtl | head -1)"
Yes, you can achieve writing to multiple files with awk which is not in the list of things you appear not to like:
echo hi | awk '{print > "a.txt"; print > "b.txt"}'
Then check a.txt and b.txt.
How to feed xargs to a piped grep for a piped cat command.
Command 1:
(Generates a grep pattern with unique PIDs for a particular date time, read from runtime.log)
cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='%
The output of above command is (It's custom grep pattern):
2018/09/13 14:50.*PID=13109
2018/09/13 14:50.*PID=14575
2018/09/13 14:50.*PID=15741
Command 2:
(Reads runtime.log and fetch the appropriate lines based on the grep pattern (Ideally the grep pattern should comes from command 1))
cat runtime.log | grep '2018/09/13 14:50.*PID=13109'
The question is How to combine both Command 1 & Command 2
Below combined version of command doesn't gives the expected output (The produced output had lines having the date other than '2018/09/13 14:50')
cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='% | cat runtime.log xargs grep
grep has an option -f. From man grep:
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file contains zero patterns, and therefore matches nothing. (-f is specified by POSIX .)
So you could use
cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='% > a_temp_file
cat runtime.log | grep -f a_temp_file
The shell has a syntax that avoids having to create the temporary file. <(). From man bash:
Process Substitution
Process substitution is supported on systems that support named pipes
(FIFOs) or the /dev/fd method of naming open files. It takes the form
of <(list) or >(list). The process list is run with its input or
output connected to a FIFO or some file in /dev/fd. The name of this
file is passed as an argument to the current command as the result of
the expansion. If the >(list) form is used, writing to the file will
provide input for list. If the <(list) form is used, the file passed
as an argument should be read to obtain the output of list.
So you can combine it to:
cat runtime.log | grep -f <(cat runtime.log | grep -e '2018/09/13 14:50' | awk -F'[ ]' '{print $4}' | awk -F'PID=' '{print $2}' | sort -u | xargs -I % echo '2018/09/13 14:50.*PID='%)
I'm using bash to look into file and parse the results. Can someone tell me how to use cut/awk to split the string and get desired output by using single command? I can get through individual cut and get the below output (with 2 commands and concatenation) but i want to do using single command instead of two commands.
test.log:
1/98 | (PASSED) com.yahoo.qa.java.projects.stackoverview.questions.Password_01() | 21:20:20
Tried code:
str1=`cat test.log | tail -1 | cut -d '|' -f 1`
str2=`cat test.log | tail -1 | cut -d '|' -f 2 | sed -e 's/com.yahoo.qa.java.projects./''/g'`
str3="${str1} | ${str2}"
Expected:
1/98 | (PASSED) stackoverview.questions.Password_01
Since this is a simple substitution on an individual line it's better suited to sed than awk and not at all appropriate for cut:
$ sed 's/\(.*| [^ ]* \)com\.yahoo\.qa\.java\.projects\.\([^(]*\).*/\1\2/' file
1/98 | (PASSED) stackoverview.questions.Password_01
Following single awk may help you in same.
awk 'END{sub(/com\.yahoo\.qa\.java\.projects\./,"",$4);print $1,$2,$3,$4}' Input_file
OR for all kind of awks following may help you in same too.(As per SIR ED's suggestions):
awk '{value=$0} END{split(value, a," ");sub(/com.yahoo.qa.java.projects\./,"",a[4]);print a[1],a[2],a[3],a[4]}' Input_file
Using awk
$ awk -F "com[.]yahoo[.]qa[.]java[.]projects[.]" 'sub(/\(\).*/,"",$2)' file
1/98 | (PASSED) stackoverview.questions.Password_01
Output of awk '{print $4}' is
b05808aa-c6ad-4d30-a334-198ff5726f7c
59996d37-9008-4b3b-ab22-340955cb6019
2b41f358-ff6d-418c-a0d3-ac7151c03b78
7ac4995c-ff2c-4717-a2ac-e6870a5670f0
I need to grep file st.log by these records. Something like awk '{print $4}' |xargs -i grep -w "pattern from awk" st.log I dont know how to pass pattern correctly?
What about
awk '{print $4}' | grep -F -f - st.log
Credits to Eric Renouf, who noticed that -f - can be used for standard input instead -f <(cat), Note: -f /dev/stdin also works and avoids launching a new process.
or closer to the question to have the output ordered
awk '{print $4}' | xargs -i grep -F {} st.log
maybe -w was not the option OP needed but -F
grep --help
-F, --fixed-strings PATTERN is a set of newline-separated strings
-w, --word-regexp force PATTERN to match only whole words
-w will match only line that contain exactly pattern
examples
grep -w . <<<a # matches
grep -w . <<<ab # doesn't match
grep -F . <<<a # doesn't match
grep -F . <<<a.b # matches
May be something along these lines be helpful
How to process each line received as a result of grep command
awk '{print $4} | while read -r line; do
grep $line st.log
done
What Linux commands would you use successively, for a bunch of files, to count the number of lines in a file and output to an output file with part of the corresponding input file as part of the output line. So for example we were looking at file LOG_Yellow and it had 28 lines, the the output file would have a line like this (Yellow and 28 are tab separated):
Yellow 28
wc -l [filenames] | grep -v " total$" | sed s/[prefix]//
The wc -l generates the output in almost the right format; grep -v removes the "total" line that wc generates for you; sed strips the junk you don't want from the filenames.
wc -l * | head --lines=-1 > output.txt
produces output like this:
linecount1 filename1
linecount2 filename2
I think you should be able to work from here to extend to your needs.
edit: since I haven't seen the rules for you name extraction, I still leave the full name. However, unlike other answers I'd prefer to use head rather then grep, which not only should be slightly faster, but also avoids the case of filtering out files named total*.
edit2 (having read the comments): the following does the whole lot:
wc -l * | head --lines=-1 | sed s/LOG_// | awk '{print $2 "\t" $1}' > output.txt
wc -l *| grep -v " total"
send
28 Yellow
You can reverse it if you want (awk, if you don't have space in file names)
wc -l *| egrep -v " total$" | sed s/[prefix]//
| awk '{print $2 " " $1}'
Short of writing the script for you:
'for' for looping through your files.
'echo -n' for printing the current file
'wc -l' for finding out the line count
And dont forget to redirect
('>' or '>>') your results to your
output file