Is it possible to use pipe output as input for grep or git grep? The data im trying to pass to grep/git grep is the following
kubectl get namespace -o name -l app.kubernetes.io/instance!=applications | cut -f2 -d "/"
argocd
default
kube-node-lease
kube-public
kube-system
nsx-system
pks-system
I've tried to extent the command but this results in an error:
kubectl get namespace -o name -l app.kubernetes.io/instance!=applications | cut -f2 -d "/" | xargs git grep -i
fatal: ambiguous argument 'default': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Using just grep results in:
kubectl get namespace -o name -l app.kubernetes.io/instance!=applications | cut -f2 -d "/" | xargs grep -i
grep: default: No such file or directory
grep: kube-node-lease: No such file or directory
grep: kube-public: No such file or directory
grep: kube-system: No such file or directory
grep: nsx-system: No such file or directory
grep: pks-system: No such file or directory
The issue im facing with grep in general in this particular case is, that even if i soley use grep within my directory, it takes ages till it's done, whereas git grep is done within seconds. If I'm not doing something terrible wrong that would explain the slow results of grep, getting git grep to work would be preferred.
I've found this other Stackoverflow Question that somewhat explains what the issue is, but I don't know how to "process" the output into git grep properly.
The problem is that (as your screenshot shows) the result is multiple terms which I'm guessing you want to be OR-ed together, and not searching for the first term in the files identified by the last terms (which is what the current xargs command does)
Since OR in regex is via the | character, you can use xargs echo to fold the vertical list into a space delimited horizontal list then replace the spaces with | and be pretty close to what you want
printf 'alpha\nbeta\ncharlie\n' | xargs echo | tr ' ' '|' | xargs git grep -i
although due to the folding operation, that command is an xargs of one line, and thus would be conceptually easier to reason about using just normal $() interpolation:
git grep -i $(printf 'alpha\nbeta\ncharlie\n' | xargs echo | tr ' ' '|')
The less "whaaa" shell pipeline would be to use kubectl get -o go-template= to actually emit a pipe-delimited list and feed that right into xargs (or $()), bypassing the need to massage the output text first
Related
I'm trying to make a shell script to monitor a log file, but I have a problem which is I can't do two tail at the same time.
the script basically is to search for a word if it matches it will redirect the 3 lines include the matched word into a file then I will pruning the useless information to extract what I want.
I tried the commands below and it's working fine but when I mirage it in a file it doesn't work
please advise :)
below is a part of the script,
#!/bin/bash
#grep error log
tail -f /FileLogging.log | grep 'error' >>/home/hello/tech.txt
#pruning useless information
tail -f /home/hello/tech.txt perl -nle "print $1 if /sam-(.+?)\",\"jack/" >>/home/hello/non.txt
Now I detected, the there only one source is watched. So this command should combine both of your example:
tail -f /FileLogging.log | grep 'error' | tee -a /home/hello/tech.txt | perl ... >>/home/hello/non.txt
I am currently learning a little more about using Bash shell on OSX terminal. I am trying to pipe the output of a cut command into a grep command, but the grep command is not giving any output even though I know there are matches. I am using the following command:
cut -d'|' -f2 <filename.txt> > <temp.txt> | grep -Ff <temp.txt> <searchfile.txt> > <filematches.txt>
I was thinking that this should work, but most of the examples I have seen normally pipe grep output into the cut. My goal was to cut field 2 from the file and use that as the pattern to search for in . However, using the command produced no output.
When I generated the temp.txt first with the cut command and then ran the grep on it manually with no pipe, the grep seemed to run fine. I am not sure why this is?
You can use process substitution here:
grep -Ff <(cut -d'|' -f2 filename.txt) searchfile.txt > filematches.txt
<(cut -d'|' -f2 filename.txt) is feeding cut command's output to grep as a file.
Okay, a reason this line doesn't behave as you expect
cut -d'|' -f2 <filename.txt> > <temp.txt> | grep -Ff <temp.txt> <searchfile.txt> > <filematches.txt>
is that the output of your cut is going to temp.txt. You're not sending anything to the pipe. Now, conveniently pipe also starts a new commend, so it doesn't matter much -- grep runs and reads searchfile.txt.
But what are you trying to do? Here's what your command line is trying to do:
take the second pipe-delimited field from filename.txt
write it to a file
run grep ...
... using the contents of the file from 2 as a grep search string (which isn't going to do what you think either, as you're effectively asking grep to look for the pattern match1\nmatch2...)
You'd be closer with
cut ... && grep ...
as that runs grep assuming cut completes effectively. Or you could use
grep -f `cut ...`
which would put the results on the command line. You need to mess with quoting, but you're still going to be looking for a line containing ALL of your match fields from cut.
I'd recommend maybe you mean something like this:
for match in `cut ...`
do
grep -f $match >> filematches.txt
done
This approach is not finding the file I think I specified.
SHELL = /bin/bash
PKG_NAME = test
PKG_VERSION := $(shell grep -i '^version' $(PKG_NAME)/DESCRIPTION | cut -d ':' -f2 | cut -d ' ' -f2)
In the shell itself, grep -i '^version' test/DESCRIPTION | cut -d ':' -f2 | cut -d ' ' -f2 does return the version successfully, e.g. 0.4-7
But, running via the makefile returns:
grep: test: Is a directory
grep: /DESCRIPTION: No such file or directory
test is a directory, that's true, but test/DESCRIPTION does exist, so I'm guessing $(PKG_NAME)/DESCRIPTION wasn't the right way to assemble the file name.
Suggestions? Thanks.
That error indicates that grep is seeing test and /DESCRIPTION as two separate arguments. Do you have extra spaces on the PKG_NAME assignment line or an errant space between $(PKG_NAME) and /DESCRIPTION in the $(shell ...) line?
As a general rule you might want to start putting quotes around arguments to shell commands (i.e. '$(PKG_NAME)/DESCRIPTION') to prevent this sort of word splitting issue (though without spaces you generally don't have that sort of problem).
This is my bash command
grep -rl "System.out.print" Project1/ |
xargs -I{} grep -H -n "System.out.print" {} |
cut -f-2 -d: |
sed "s/\(.*\):\(.*\)/filename is \1 and line number is \2/
What I'm trying to do here is,I'm trying to iterate through sub folders and check what files contains "System.out.print" (using grep)
using 2nd grep trying to get file names and line numbers
using sed command I display those to console.
from here I want to remove "System.out.print" with "XXXXX" how I can pipe sed command to this?
pls help me
thanxx
GNU sed has an option to change files in place:
find Project1/ -type f | xargs sed -i 's/System\.out\.print/XXXXX/g'
Btw, your script could be written as:
grep -rsn 'root' /etc/ |
awk -F: '{ print "filename is", $1, "and line number is", $2 }'
I'm just building on hop's answer, which I found to be more useful than find -exec. I had search_text dispersed all over my computer, in logs, config files and so on, but I didn't want to search (or especially change) anything in /dev, /sys, /proc, and so on. One note, read man xargs; it doesn't like file names with spaces.
grep -HriIl --exclude-dir=dev --exclude-dir=proc --exclude-dir=sys search_text / | xargs sed -i 's/search_text/replace_text/g'
I am using 'tail -f' to follow a log file as it's updated; next I pipe the output of that to grep to show only the lines containing a search term ("org.springframework" in this case); finally I'd like to make is piping the output from grep to a third command, 'cut':
tail -f logfile | grep org.springframework | cut -c 25-
The cut command would remove the first 25 characters of each line for me if it could get the input from grep! (It works as expected if I eliminate 'grep' from the chain.)
I'm using cygwin with bash.
Actual results: When I add the second pipe to connect to the 'cut' command, the result is that it hangs, as if it's waiting for input (in case you were wondering).
Assuming GNU grep, add --line-buffered to your command line, eg.
tail -f logfile | grep --line-buffered org.springframework | cut -c 25-
Edit:
I see grep buffering isn't the only problem here, as cut doesn't allow linewise buffering.
you might want to try replacing it with something you can control, such as sed:
tail -f logfile | sed -u -n -e '/org\.springframework/ s/\(.\{0,25\}\).*$/\1/p'
or awk
tail -f logfile | awk '/org\.springframework/ {print substr($0, 0, 25);fflush("")}'
On my system, about 8K was buffered before I got any output. This sequence worked to follow the file immediately:
tail -f logfile | while read line ; do echo "$line"| grep 'org.springframework'|cut -c 25- ; done
What you have should work fine -- that's the whole idea of pipelines. The only problem I see is that, in the version of cut I have (GNU coreutiles 6.10), you should use the syntax cut -c 25- (i.e. use a minus sign instead of a plus sign) to remove the first 24 characters.
You're also searching for different patterns in your two examples, in case that's relevant.