How can I comment out lines where a certain word can be found in a bash script, using piped UNIX commands (no sed/awk) ?
The comment character is # .
Here is how It could start :
cat $file | grep $word | ...
With GNU bash.
#!/bin/bash
keyword="foo"
while IFS= read -r line; do
[[ "$line" =~ $keyword ]] && line="${line#*#}"
printf "%s\n" "$line"
done < /etc/network/interfaces > /tmp/interfaces_modified
Related
so the file contains data like
# entertainment
youtube.com
twitch.tv
# research
google.com
wikipedia.com
...
and I would like to pass that file as an argument in a script that would open all lines if they doesn't start with an #. Any clues on how to ?
so far what i have:
for Line in $Lines
do
case "# " in $Line start firefox $Line;; esac
done
some code that could be useful (?):
while read line; do chmod 755 "$line"; done < file.txt
grep -e '^[^#]' inputfile.txt | xargs -d '\n' firefox --new-tab
grep -e '^[^#]': Will print all lines that don't start with a sharp (comments)
xargs -d '\n' firefox --new-tab: Will pass each line that is not blank, as argument to Firefox.
Removes both the lines that start with # and empty lines.
#!/bin/bash
#
while read -r line
do
if [[ $(echo "$line" | grep -Ev "^#|^$") ]]
then
firefox --new-tab "$url" &
fi
done <file.txt
Skip the empty lines and the lines that starts with a #
#!/usr/bin/env bash
while IFS= read -r url; do
[[ "$url" == \#* || -z "$url" ]] && continue
firefox --new-tab "$url" &
done < file.txt
awk 'NF && $1!="#"{print "firefox --new-tab", $0, "&"}' file.txt | bash
I have a text file which is the output difference of two grepped files . the text file has lines like below I need to read the file (loop through the lines in the text file ) and based on text to the left hand side of '<' and right hand side of '>' do something.
editing to add details:
LHS of < OR RHS of >
if either of those, i will need to store the content into a variable, and get the 1st(ABCDEF) 3rd(10) and search (will grep) for them in one of other two files and if found print a message and attach those file(s) names in an email DL. All the file names and directories have been stored in separate variables.
how do i do that.
ps:have basic knowledge on text formatting and bash/shell commands but still learning the scripting syntax.Thanks.
ABCDEF,20200101,10 <
PQRSTU,20200106,11 <
LMNOPQ,20200101,12 <
EFGHIJ,20200102,13 <
KLMNOP,20200103,14 <
STUVWX,20200104,15 <
PQRSTU,20200105,16 <
> LMNOPQ,20200101,10
ABCDEF,20200107,17 <
What wrong am I doing now?
while IFS= read -r line; do
if $line =~ ([^[:blank:]]+)[[:blank:]]+\<
then
IFS=, read -r f1 f2 f3 <<< "${BASH_REMATCH[1]}"
#echo "f1=$f1 f2=$f2 f3=$f3"
zgrep "$f1" file1 | grep "with seq $f3" || zgrep "$f1" file2 | grep "with seq $f3"
elif $line =~ \>[[:blank:]]+([^[:blank:]]+)
then
IFS=, read -r g1 g2 g3 <<< "${BASH_REMATCH[1]}"
#echo "g1=$g1 g2=$g2 g3=$g3"
zgrep "$g1" file3 | grep "with seq $g3" || zgrep "$g1" file3 | grep "with seq $g3"
fi
Would you please try something like:
#!/bin/bash
while IFS= read -r line; do
if [[ $line =~ ([^[:blank:]]+)[[:blank:]]+\< || $line =~ \>[[:blank:]]+([^[:blank:]]+) ]]; then
IFS=, read -r f1 f2 f3 <<< "${BASH_REMATCH[1]}"
echo "f1=$f1 f2=$f2 f3=$f3"
# do something here with "$f1", "$f2" and "$f3"
fi
done < file.txt
Output:
f1=ABCDEF f2=20200101 f3=10
f1=PQRSTU f2=20200106 f3=11
f1=LMNOPQ f2=20200101 f3=12
f1=EFGHIJ f2=20200102 f3=13
f1=KLMNOP f2=20200103 f3=14
f1=STUVWX f2=20200104 f3=15
f1=PQRSTU f2=20200105 f3=16
f1=LMNOPQ f2=20200101 f3=10
f1=ABCDEF f2=20200107 f3=17
Please modify the echo "f1=$f1 f2=$f2 f3=$f3" line to your desired
command such as grep.
The regex ([^[:blank:]]+)[[:blank:]]+\< matches a line which contains <
and assigns the bash variable ${BASH_REMATCH[1]} to the LHS.
On the other hand, the regex \>[[:blank:]]+([^[:blank:]]+) does the similar thing for
a line which contains >.
The statement IFS=, read -r f1 f2 f3 <<< "${BASH_REMATCH[1]}" splits the bash variable
on , and assigns f1, f2 and f3 to the fields.
Please note if the input file is very large, bash solution may not
be efficient in execution time. I used bash just because it will be convenient
to pass the variables to your grep command.
EDIT
Regarding the updated script in your question, please refer to the following modification:
while IFS= read -r line; do
if [[ $line =~ ([^[:blank:]]+)[[:blank:]]+\< ]]; then
IFS=, read -r f1 f2 f3 <<< "${BASH_REMATCH[1]}"
# echo "f1=$f1 f2=$f2 f3=$f3"
result=$(zgrep "$f1" file1 | grep "with seq $f3" || zgrep "$f1" file2 | grep "with seq $f3")
elif [[ $line =~ \>[[:blank:]]+([^[:blank:]]+) ]]; then
IFS=, read -r g1 g2 g3 <<< "${BASH_REMATCH[1]}"
# echo "g1=$g1 g2=$g2 g3=$g3"
result=$(zgrep "$g1" file3 | grep "with seq $g3" || zgrep "$g1" file3 | grep "with seq $g3")
fi
if [[ -n $result ]]; then
echo "result = $result"
fi
done < file.txt
I have the next two regex in Bash:
1.^[-a-zA-Z0-9\,\.\;\:]*$
2.^[]a-zA-Z0-9\,\.\;\:]*$
The first matches when the string contains a "-" and the other values.
The second when contains a "]".
I put this values at the beginning of my regex because I can't scape them.
How I can get match the two values at the same time?
You can also place the - at the end of the bracket expression, since a range must be closed on both ends.
^[]a-zA-Z0-9,.;:-]*$
You don't have to escape any of the other characters, either. Colons, semicolons, and commas have no special meaning in any part of a regular expression, and while a period loses its special meaning inside a bracket expression.
Basically you can use this:
grep -E '^.*\-.*\[|\[.*\-.*$'
It matches either a - followed by zero or more arbitrary chars and a [ or a [ followed by zero or more chars and a -
However since you don't accept arbitrary chars, you need to change it to:
grep -E '^[a-zA-Z0-9,.;:]*\-[a-zA-Z0-9,.;:]*\[|\[[a-zA-Z0-9,.;:]*\-[a-zA-Z0-9,.;:]*$'
Maybe, this can help you
#!/bin/bash
while read p; do
echo $p | grep -E '\-.*\]|\].*\-' | grep "^[]a-zA-Z0-9,.;:-]*$"
done <$1
user-host:/tmp$ cat test
-i]string
]adfadfa-
string-
]string
str]ing
]123string
123string-
?????
++++++
user-host:/tmp$ ./test.sh test
-i]string
]adfadfa-
There are two questions in your post.
One is in the description:
How I can get match the two values at the same time?
That is an OR match, which could be done with a range that mix your two ranges:
pattern='^[]a-zA-Z0-9,.;:-]*$'
That will match a line that either contains one (or several) -…OR…]…OR any of the included characters. That would be all the lines (except ?????, ++++++ and as df gh) in the test script below.
Two is in the title:
… a string contains “-” and “]” at the same time
That is an AND match. The simplest (and slowest) way to do it is:
echo "$line" | grep '-' | grep ']' | grep '^[-a-zA-Z0-9,.;:]*$'
The first two calls to grep select only the lines that:
contain both (one or several) - and (one or several) ]
Test script:
#!/bin/bash
printlines(){
cat <<-\_test_lines_
asdfgh
asdfgh-
asdfgh]
as]df
as,df
as.df
as;df
as:df
as-df
as]]]df
as---df
asAS]]]DFdf
as123--456DF
as,.;:-df
as-dfg]h
as]dfg-h
a]s]d]f]g]h
a]s]d]f]g]h-
s-t-r-i-n-g]
as]df-gh
123]asdefgh
123asd-fgh-
?????
++++++
as df gh
_test_lines_
}
pattern='^[]a-zA-Z0-9,.;:-]*$'
printf '%s\n' "Testing the simple pattern of $pattern"
while read line; do
resultgrep="$( echo "$line" | grep "$pattern" )"
printf '%13s %-13s\n' "$line" "$resultgrep"
done < <(printlines)
echo "#############################################################"
echo
p1='-'; p2=']'; p3='^[]a-zA-Z0-9,.;:-]*$'
printf '%s\n' "Testing a 'grep AND' of '$p1', '$p2' and '$p3'."
while read line; do
resultgrep="$( echo "$line" | grep "$p1" | grep "$p2" | grep "$p3" )"
[[ $resultgrep ]] && printf '%13s %-13s\n' "$line" "$resultgrep"
done < <(printlines)
echo "#############################################################"
echo
printf '%s\n' "Testing an 'AWK AND' of '$p1', '$p2' and '$p3'."
while read line; do
resultawk="$( echo "$line" |
awk -v p1="$p1" -v p2="$p2" -v p3="$p3" '$0~p1 && $0~p2 && $0~p3' )"
[[ $resultawk ]] && printf '%13s %-13s\n' "$line" "$resultawk"
done < <(printlines)
echo "#############################################################"
echo
printf '%s\n' "Testing a 'bash AND' of '$p1', '$p2' and '$p3'."
while read line; do
rgrep="$( echo "$line" | grep "$p1" | grep "$p2" | grep "$p3" )"
[[ ( $line =~ $p1 ) && ( $line =~ $p2 ) && ( $line =~ $p3 ) ]]
rbash=${BASH_REMATCH[0]}
[[ $rbash ]] && printf '%13s %-13s %-13s\n' "$line" "$rgrep" "$rbash"
done < <(printlines)
echo "#############################################################"
echo
I have this block of code:
while IFS=$'\n' read -r line || [[ -n "$line" ]]; do
if [ "$line" != "" ]; then
echo -e "$lanIP\t$line" >> /tmp/ipList;
fi
done < "/tmp/includeList"
I know this must be really simple. But I have another list (/tmp/excludeList). I only want to echo the line within my while loop if the line ins't found in my excludeList. How do I do that. Is there some awk statement or something?
You can do this with grep alone:
$ cat file
blue
green
red
yellow
pink
$ cat exclude
green
pink
$ grep -vx -f exclude file
blue
red
yellow
The -v flag tells grep to only output the lines in file that are not found in exclude and the -x flags forces whole line matching.
use grep
while IFS=$'\n' read -r line || [[ -n "$line" ]]; do
if [[ -n ${line} ]] \
&& ! grep -xF "$line" excludefile &>/dev/null; then
echo -e "$lanIP\t$line" >> /tmp/ipList;
fi
done < "/tmp/includeList"
the -n $line means if $line is not empty
the grep returns true if $line is found in exclude file which is inverted by the ! so returns true if the line is not found.
-x means line matched so nothing else can appear on the line
-F means fixed string so if any metacharacters end up in $line they'll be matched literally.
Hope this helps
With awk:
awk -v ip=$lanIP -v OFS="\t" '
NR==FNR {exclude[$0]=1; next}
/[^[:space:]]/ && !($0 in exclude) {print ip, $0}
' /tmp/excludeList /tmp/includeList > /tmpipList
This reads the exclude list info an array (as the array keys) -- the NR==FNR condition is true while awk is reading the first file from the arguments. Then, while reading the include file, if the current line contains a non-space character and it does not exist in the exclude array, print it.
The equivalent with grep:
grep -vxF -f /tmp/excludeList /tmp/includeList | while IFS= read -r line; do
[[ -n "$line" ]] && printf "%s\t%s\n" "$ipList" "$line"
done > /tmp/ipList
I am trying to write a script which reads a text file and saves each line to a string. I would also like the script to skip any lines which start with a hash symbol. Any suggestions?
You should not leave skipping lines to ksh. E.g. do this:
grep -v '^#' INPUTFILE | while IFS="" read line ; do echo $line ; done
And instead of the echo part do whatever you want.
Or if ksh does not support this syntax:
grep -v '^#' INPUTFILE > tmpfile
while IFS="" read line ; do echo $line ; done < tmpfile
rm tmpfile
while read -r line; do
[[ "$line" = *( )#* ]] && continue
# do something with "$line"
done < filename
look for "File Name Patterns" or "File Name Generation" in the ksh man page.