I am trying to combine grep and grep -v search together.
Output should be display all lines ending with .xml, but to exclude lines starting with $.
Here are the commands I have tried; none worked:
grep *.xml file1.txt | grep -v '$' file1.txt > output
grep *.xml | grep -v '$' file1.txt > output
grep *.xml grep -v '$' file1.txt > output
grep *.xml '$' file1.txt > output
To match a $ at the start of a line, anchor it to the start of the line with ^. Also, $ by itself matches the end of the line (it's a special character, just like ^), and * will not do what you think it does (it works differently in regular expressions compared to in shell globbing patterns). So,
grep -v '^\$'
will filter out all lines starting with a $.
You can do either
grep '\.xml$' file1.txt | grep -v '^\$'
or
grep '^[^$].*\.xml$' file1.txt
to find all lines in the file file1.txt that do not start with $ but that ends with .xml.
Notice that I also escape the dot in .xml as that otherwise matches any character, and that the second command combines both criteria by using a character range ([ ... ]) containing all characters except $ (the .* matches any number of any characters).
The single quotes are necessary so that the shell won't interpret the regular expression as a shell globbing pattern.
You should use "cat" command to direct the output to an file.
And then use regular expression to filter the keyword, in this case all lines start with $ symbol is '^[$]'.
So you can use command cat *.xml | grep -v '^[$]'.
Related
I have a file named "TestGrep" that contains content as shown below
#!/bin/bash
/ParentFolder/a #email1.com
/ParentFolder/b #email2.com
/ParentFolder/.a #email1.com
/ParentFolder/.b #email2.com
/ParentFolder/ #email3.com
I am using the below grep command
grep -Fw "/ParentFolder/" TestGrep
The output is
/ParentFolder/.a #email1.com
/ParentFolder/.b #email2.com
/ParentFolder/ #email3.com
It is somehow ignoring the dots in the TestGrep file.
I want the output to be shown as below
/ParentFolder/ #email3.com
How can I query using grep command that would just check if the exact string match is done and return output as expected.
Could you please try following. Using -E option of grep here.
grep -E '/ParentFolder/\s+' Input_file
From man grep about -E option of grep:
-E, --extended-regexp
Interpret PATTERN as an extended regular expression
\s+ means looks for spaces one or more occurrences.
I have the below example file
d41d8cd98f00b204e9800998ecf8427e /home/abid/Testing/FileNamesTest/apersand $ file
d41d8cd98f00b204e9800998ecf8427e /home/abid/Testing/FileNamesTest/file[with square brackets]
d41d8cd98f00b204e9800998ecf8427e /home/abid/Testing/FileNamesTest/~$tempfile
017a3635ccb76250b2036d6aea330c80 /home/abid/Testing/FileNamesTest/FileThree
217a3635ccb76250b2036d6aea330c80 /home/abid/Testing/FileNamesTest/FileThreeDays
d41d8cd98f00b204e9800998ecf8427e /home/abid/Testing/FileNamesTest/single quote's
I want to grep the last part of the file (the file name) but I'm after an exact match for the last part of the line (the file name)
grep FileThree$ files.md5
017a3635ccb76250b2036d6aea330c80 /home/abid/Testing/FileNamesTest/FileThree
gives back an exact match and doesnt find "FileThreeDays" which is what I'm after but because some of the file names contains square brackets it I'm having to use grep -F or fgrep. However using fgrep like the above doesnt work it returns nothing.
How can I exact match the last part of the line using fgrep whilst still honoring the special characters above ~ / $ / ' / [ ] etc...or any other method using maybe awk...
Further....
using fgrep withou return both these files I only want an exact match (using the use of the $ above with grep), but $ with fgrep doesnt return anything.
grep -F FileThree files.md5
017a3635ccb76250b2036d6aea330c80 /home/abid/Testing/FileNamesTest/FileThree
217a3635ccb76250b2036d6aea330c80 /home/abid/Testing/FileNamesTest/FileThreeDays
I can't tell all the details from your question, but it sounds like you can use grep and just escape the special characters: grep 'File\[Three\]Days$'
If you want to use fgrep, though, you can use some tr tricks to help you. If all you want is the filename (without the directory name), you can do something like
cat files.md5 | tr '/' '\n' | fgrep FileThreeDays
That tr command replaces slashes with newlines, so it will put each filename on its own line. That means that fgrep will only find the filename when it searches for FileThreeDays.
If you want the full filename with directory, it's a little trickier, but a similar approach will work. Assuming that there's always a double space between the SHA and the filename, and that there aren't any filenames with double spaces or tab characters in them, you can try something like this:
sed 's/ /\t' files.md5 | tr '\t' '\n' | fgrep FileThreeDays
That sed command converts the double spaces to tabs. The tr command turns those tabs into newlines (the same trick as above).
I would use awk:
awk '{$1="";print}' file
$1="" cuts the first column to an empty string, and print prints the modified line - which only contains the filename now.
However, this leaves a blank space at the start of each line. If you care about it and want to remove it, set the output field separator to an empty string:
awk '{$1="";print}' OFS="" file
I have a directory with contents like this -
vishal.yadav#droid36:~/Shell$ ls
lazy_dog.txt ls-error.txt ls-output.txt ShellCommands.txt TheTimeMachineHGWells.txt words.txt words.txt.bak
First Command
If I try using ls | grep *.txt I get the following output -
ShellCommands.txt: $ cat > lazy_dog.txt
ShellCommands.txt: $ cat lazy_dog.txt
ShellCommands.txt: $ cat < lazy_dog.txt
ShellCommands.txt:input from the keyboard to the file lazy_dog.txt. We see that the result is the
Second Command
And if I use ls | grep .*.txt I get this as output -
lazy_dog.txt
ls-error.txt
ls-output.txt
ShellCommands.txt
TheTimeMachineHGWells.txt
words.txt
words.txt.bak
Isn't .*.txt and *.txt one and the same?
In the First Command, is the output of ls the regex for grep or is it the list of files?
Similarly, for the Second Command, is the output of ls the regex or list of files?
In the first command (ls | grep *.txt), the output from ls is completely ignored by grep because it sees:
grep lazy_dog.txt ls-error.txt ls-output.txt ShellCommands.txt TheTimeMachineHGWells.txt
It has one pattern lazy_dog.txt and four files, so it reads each file in turn to find the pattern, and prefixes the matching output lines with the name of the file that held the pattern. If there was only one file name, it would not list the file name before the matched lines.
It appears that the only file of the four that grep searches (ls-error.txt, ls-output.txt, ShellCommands.txt, TheTimeMachineHGWells.txt) that contains the text lazy_dog.txt is ShellCommands.txt, so that's what you see in the output. Note that a line containing lazy_dogstxt would also match the regex (but not the shell glob).
In the second command (ls | grep .*.txt), there are no files that match .*.txt, so that argument is passed to grep unexpanded, so it has only a pattern, so it reads its standard input, which is the output from ls this time. All the file names match the regex .*.txt (even though none of them match the shell glob .*.txt), so they're all listed. Note that it would also pick up many other lines, even one containing just "etxt", because the . is a grep metacharacter (and the .*.txt regex matches any string of zero or more characters followed by one arbitrary character and then txt.
do ls -al:
you will find that the current directory is listed as a . and previous directory is listed as ...
So when you say ls | grep .*.txt, the . is taken as path matching from current directory that contains .txt afterwards.
grep see the pattern .*.txt as regex, not glob.
So you can use ls *.txt or ls | grep .*txt
I have a file temp.txt as below.
a.*,super
I want to grep .* to check whether the value is present in the file or not.
Command used:
grep -i ".*" temp.txt
returns nothing
This is because grep considers the pattern as a regular expression.
To make grep interpret it as a literal, use -F.
grep -F ".*" temp.txt
Also, note -i is not needed, because there is no case distinction to take into account (we for example use it to make grep return AB, aB, Ab and ab when doing grep -i "ab").
As man grep says:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines,
any of which is to be matched. (-F is specified by POSIX.)
-i, --ignore-case
Ignore case distinctions in both the PATTERN and the input files. (-i
is specified by POSIX.)
Using awk
awk '/\.\*/' file
or fgrep
fgrep ".*" file
Both ., * have special meaning in regular expression. Escape them to match literally.
$ cat temp.txt
a.*,super
$ grep "\.\*" temp.txt
a.*,super
$ echo $?
0
$ grep "there-is-no-such-string" temp.txt
$ echo $?
1
-i is not need because there's no alphabet in the regular expression.
I have a text file that's about 300KB in size. I want to remove all lines from this file that begin with the letter "P". This is what I've been using:
> cat file.txt | egrep -v P*
That isn't outputting to console. I can use cat on the file without another other commands and it prints out fine. My final intention being to:
> cat file.txt | egrep -v P* > new.txt
No error appears, it just doesn't print anything out and if I run the 2nd command, new.txt is empty.
I should say I'm running Windows 7 with Cygwin installed.
Explanation
use ^ to anchor your pattern to the beginning of the line ;
delete lines matching the pattern using sed and the d flag.
Solution #1
cat file.txt | sed '/^P/d'
Better solution
Use sed-only:
sed '/^P/d' file.txt > new.txt
With awk:
awk '!/^P/' file.txt
Explanation
The condition starts with an ! (negation), that negates the following pattern ;
/^P/ means "match all lines starting with a capital P",
So, the pattern is negated to "ignore lines starting with a capital P".
Finally, it leverage awk's behavior when { … } (action block) is missing, that is to print the record validating the condition.
So, to rephrase, it ignores lines starting with a capital P and print everything else.
Note
sed is line oriented and awk column oriented. For your case you should use the first one, see Edouard Lopez's reponse.
Use sed with inplace substitution (for GNU sed, will also for your cygwin)
sed -i '/^P/d' file.txt
BSD (Mac) sed
sed -i '' '/^P/d' file.txt
Use start of line mark and quotes:
cat file.txt | egrep -v '^P.*'
P* means P zero or more times so together with -v gives you no lines
^P.* means start of line, then P, and any char zero or more times
Quoting is needed to prevent shell expansion.
This can be shortened to
egrep -v ^P file.txt
because .* is not needed, therefore quoting is not needed and egrep can read data from file.
As we don't use extended regular expressions grep will also work fine
grep -v ^P file.txt
Finally
grep -v ^P file.txt > new.txt
This works:
cat file.txt | egrep -v -e '^P'
-e indicates expression.