bash strip line matching wildcard - bash

sed -i -e "/^*google.com*/d" activedomains.txt
What I am trying to do is strip any line containing * google.com * it needs to be the wildcard on both front and rear, can't seem to figure it out :/

sed uses regex, not globbing (although maybe there is something that does). Pretty simple to change, though:
sed -i '/google\.com/d' activedomains.txt
This deletes any line that matches google.com. You could also use
sed -i -e '/^.*google.com.*/d' activedomains.txt
...which is more and line with what you were doing and literally means "the start of the string, then zero or more of any character followed by 'google (one of any character) com' followed by zero or more of any character." Of course, since it is surrounded by "zero or mores," it's just as well to match it directly.

do you mean this?
sed -i -e "/google\.com/d" activedomains.txt

This should works :
sed -i -e "/google.com/d" activedomains.txt
No need wildcard here : it's like a grep

Related

replace recursively shabang with sed

I'm trying to replace recursively all shabang from a folder (for run program in android..), with sed .
The command works good when i tried with "normal word" but become a headache when i'm trying with shabang..Everything i tried, I got error :
bad option in substitution expression
unmatched '/'
event not found
I'm new to this and it's probably begginer mistake, so here is the code (tsst is my folder):
grep -rl "env python" tsst |xargs sed -i "s/\#!/usr/bin/env python/\#!/system/python2.7.9//g"
I also tried with variables:(first part of code is good .. i just copy where there's a problem.)
sed -i "s/$old/$new/g"
sed -i 's/"$old"/"$new"/g'
sed -i "s/'\#!\/usr\/bin\/env python'/'\#!\/system\/python2.7.9\/'/g"
What did i do wrong ?
Try this regex. You can't mix / as the regex delimiter and actual characters you want to match. You can use any character as the regex delimiter, such as |, as long as you use it instead of the / in all 3 spots.
$ echo '#!/usr/bin/env python' | sed 's|#!/usr/bin/env python|#!/system/python2.7.9|g'
#!/system/python2.7.9

Replacing a pattern using sed

I am trying to replace all the patterns
s#_coded_block[#] with s#_coded_block_# in myfile. I looked online on how to replace patterns with groupings and my command is:
sed -i -E 's/s\([0-9]*\)_coded_block\[\([0-9]*\)\]/s\1_coded_block_\2/g' myfile
However, I am getting
invalid reference \2 on `s'
command's RHS when I execute this command.
With the -E option, you don't need backslashes before the capturing parentheses:
sed -i -E 's/s([0-9]*)_coded_block\[([0-9]*)\]/s\1_coded_block_\2/g' myfile
You might want one-or-more digits, in which case you use + instead of *. If you decide to drop the -E, your original code should work, though if you want at least one digit, you need to write \{1,\}:
sed -i 's/s\([0-9]\{1,\}\)_coded_block\[\([0-9]\{1,\}\)\]/s\1_coded_block_\2/g' myfile
The -i notation shown only works reliably with GNU sed. BSD (macOS or Mac OS X) sed would treat the -E in the first command line as the suffix (in the second, you'd get a complaint about m not being a valid sed command because the script would be treated as the suffix and the m of myfile would be an erroneous sed command. You'd use -i '' to back up (overwrite) a file with no suffix. If you want portable code, use -i.bak which creates a backup file with both variants — the .bak must be attached to the -i for GNU sed.

How to grep information?

What I have:
test
more text
#user653434 text and so
test
more text
#user9659333 text and so
I'd like to filter this text and finally get the following list as .txt file:
user653434
user9659333
It's important to get the names without "#" sign.
Thx for help ;)
Using grep -P (requires GNU grep):
$ grep -oP '(?<=#)\w+' File
user653434
user9659333
-o tells grep to print only the match.
-P tells grep to use Perl-style regular expressions.
(?<=#) tells sed that # must precede the match but the # is not included in the match.
\w+ matches one or more word characters. This is what grep will print.
To change the file in place with grep:
grep -oP '(?<=#)\w+' File >tmp && mv tmp File
Using sed
$ sed -En 's/^#([[:alnum:]]+).*/\1/p' File
user653434
user9659333
And, to change the file in place:
sed -En -i.bak 's/^#([[:alnum:]]+).*/\1/p' File
-E tells sed to use the extended form of regular expressions. This reduces the need to use escapes.
-n tells sed not to print anything unless we explicitly ask it to.
-i.bak tells sed to change the file in place while leaving a backup file with the extension .bak.
The leading s in s/^#([[:alnum:]]+).*/\1/p tells sed that we are using a substitute command. The command has the typical form s/old/new/ where old is a regular expression and sed replaces old with new. The trailing p is an option to the substitute command: the p tells sed to print the resulting line.
In our case, the old part is ^#([[:alnum:]]+).*. Starting from the beginning of the line, ^, this matches # followed by one or more alphanumeric characters, ([[:alnum:]]+), followed by anything at all, .*. Because the alphanumeric characters are placed in parens, this is saved as a group, denoted \1.
The new part of the substitute command is just \1, the alphanumeric characters from above which comprise the user name.
Here, the s indicates that we are using a sed substitute command. The usual form
With GNU grep:
grep -Po '^#\K[^ ]*' file
Output:
user653434
user9659333
See: The Stack Overflow Regular Expressions FAQ

How to pass special characters through sed

I want to pass this command in my script:
sed -n -e "/Next</a></p>/,/Next</a></p>/ p" file.txt
This command (should) extract all text between the two matched patterns, which are both Next</a></p> in my case. However when I run my script I keep getting errors. I've tried:
sed -n -e "/Next\<\/a\>\<\/p\>/,/Next<\/a\>\<\/p>/ p" file.txt with no luck.
I believe the generic pattern for this command is this:
sed -n -e "/pattern1/,/pattern2/ p" file.txt
I can't get it working for Next</a></p> though and I'm guessing it has something to do with the special characters I am encasing. Is there any way to pass Next</a></p> in the sed command? Thanks in advance guys! This community is awesome!
You don't need to use / as a regular expression delimiter. Using a different character will make quoting issues slightly easier. The syntax is
\cregexc
where c can be any character (other than \) that you don't use in the regex. In this case, : might be a good choice:
sed -n -e '\:Next</a></p>:,\:Next</a></p>: p' file.txt
Note that I changed " to ' because inside double quotes, \ will be interpreted by bash as an escape character, whereas inside single quotes \ is just treated as a regular character. Consequently, you could have written the version with escaped slashes like this:
sed -n -e '/Next<\/a><\/p>/,/Next<\/a><\/p>/ p' file.txt
but I think the version with colons is (slightly) easier to read.
You need to escape the forward slashes inside the regular expressions with a \, since the forward slashes serve as delimiters for the regexes
sed -n -e '/Next<\/a><\/p>/,/Next<\/a><\/p>/p' file.txt

Replacing "#", "$", "%", "&", and "_" with "\#", "\$", "\%", "\&", and "\_"

I have a plain text document, which I want to compile inside LaTeX. However, sometimes it has the characters, "#", "$", "%", "&", and "_". To compile properly in LaTeX, I must first replace these characters with "#", "\$", "\%", "\&", and "_". I have used this line in sed:
sed -i 's/\#/\\\#/g' ./file.txt
sed -i 's/\$/\\\$/g' ./file.txt
sed -i 's/\%/\\\%/g' ./file.txt
sed -i 's/\&/\\\&/g' ./file.txt
sed -i 's/\_/\\\_/g' ./file.txt
Is this correct?
Unfortunately, the file is too large to open in any GUI software, so checking if my sed line is correct with a text editor is difficult. I tried searching with grep, but the search does not work as expected (e.g. below, I searched for any lines containing "$"):
grep "\$" file.txt
What is the best way to put "\" in front of these characters?
How can I use grep to successfully check the lines with the replacements?
You can do the replacement with a single call to sed:
sed -i -E 's/([#$%&_\])/\\&/g' file.txt
The & in the replacement text fills in for whichever single character is enclosed in parentheses. Note that since \ is the LaTeX escape character, you'll have to escape it as well in the original file.
sed -i 's/\#/\\\#/g' ./file.txt
sed -i 's/\$/\\\$/g' ./file.txt
sed -i 's/\%/\\\%/g' ./file.txt
sed -i 's/\&/\\\&/g' ./file.txt
sed -i 's/\_/\\\_/g' ./file.txt
You don't need the \ on the first (search) string on most of them, just $ (it's a special character, meaning the end of a line; the rest aren't special). And in the replacement, you only need two \\, not three. Also, you could do it all in one with several -e statements:
sed -i.bak -e 's/#/\\#/g' \
-e 's/\$/\\$/g' \
-e 's/%/\\%/g' \
-e 's/&/\\&/g' \
-e 's/_/\\_/g' file.txt
You don't need to double-escape anything (except the \\) because these are single-quoted. In your grep, bash is interpreting the escape on the $ because it's a special character (specifically, a sigil for variables), so grep is getting and searching for just the $, which is a special character meaning the end of a line. You need to either single-quote it to prevent bash from interpreting the \ ('\$', or add another pair of \\: "\\\$". Presumably, that's where you're getting the\` from, but you don't need it in the sed as it's written.
I think your problem is that bash itself is handling those escapes.
What you have looks right to me. But warning: it will also doubly escape e.g. a \# that is already escaped. If that's not what you want, you might want to modify your patterns to check that there isn't a preceding \ already.
$ is used for bash command substitution syntax. I guess grep "\\$" file.txt should do what you expect.
I do not respond for sed, the other answers are good enougth ;-)
You can use less as viewer to check your huge file (or more, but less is more comfortable than more).
For searching, you can use fgrep: it ignores regular expression => fgrep '\$' will really search for text \$. fgrep is the same as invoking grep -F.
EDIT:
fgrep '\$' and fgrep "\$" are different. In the second case, bash interprets the string and will replace it by a single character: $ (i.e. fgrep will search for $ only).

Resources