How can I use bash to scan for a specific string of characters in a text file and then use an if then statement to execute a specific command depending on the string?
I am trying to use rsync to back up some raspberry pis from a query string output from an HTML form. I have minimal bash experience and I have been poking and prodding this code for days now and I would love for some advice.
QUERY_STRING will contain something similar to "Backup_To_Comp=tasting-side_backup&subbtn=Submit" with the "Tasting-side_backup" being swapped for other radial button tags as selected.
#!/bin/bash
echo "Content-type: text/html"
echo ""
echo "$QUERY_STRING" > /var/www/cgi-bin/scan.txt
BackupToCompFrom=`echo "$QUERY_STRING" | sed -n 's/^.*Backup_To_Comp=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
echo "<html><head><title>What You Said</title></head>"
echo "<body>Here's what you said:"
echo "You entered $BackupToCompFrom in from field."
sleep 1
file="/var/www/cgi-bin/scan.txt"
##echo "$QUERY_STRING"
while IFS='' read -r line || [[ -n "$line" ]];
do
if [[ $line = "tasting-side_backup" ]]; then
echo "GotIt."
rsync pi#192.168.1.1:/home/pi/screenly_assets /home/pi/Downloads
elif [[ $line = "~tasting-main"* ]]; then
print "Tasting Main"
elif [[ $line = "~lodge"* ]]; then
print "Lodge"
elif [[ $line = "~barn"* ]]; then
print "Barn"
else
print "Please select a pi to copy from!"
fi
done
How can I use bash to scan for a specific string of characters in a text file and then use an if then statement to execute a specific command depending on the string?
You can use a command in the if statement. Now the exit code for that command is used to determine true or false. For simple searching of a query to a file, you can use grep.
if grep -q "$QUERY_STRING" file; then
The -q is used to prevent any output from grep ending up in stdin.
Related
I'm trying to automate a build process by replacing .js chunks for particular lines in my main.config.php file. When I run the following code:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
sed -i "s/$line/$line$(basename $js_file)\";/g" $config_path
fi
done
fi
done
done < "$config_path"
I get this message back, and file has not been updated/edited:
sed: 1: "./system/Config/main.co ...": invalid command code .
I haven't been able to find anything in my searches that pertain to this specific message. Does anyone know what I need to change/try to get the specific lines replaced in my .php file?
Updated script with same message:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
filename=$(basename $js_file)
newline="${line//$js_string*/$filename\";}"
echo $line
echo $newline
sed -i "s\\$line\\$newline\\g" $config_path
echo ""
fi
done
fi
done
done < "$config_path"
Example $line:
$config['public_build_header_css_url'] = "http://localhost:8080/build/headerCSS.js";
Example $newline:
$config['public_build_header_css_url'] = "http://localhost:8080/build/headerCSS.7529a73071877d127676.js";
Updated script with changes suggested by #Vercingatorix:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
filename=$(basename $js_file)
newline="${line//$js_string*/$filename\";}"
echo $line
echo $newline
linenum=$(grep -n "^${line}\$" ${config_path} | cut -d':' -f 1 )
echo $linenum
[[ -n "${linenum}" ]] && sed -i "${linenum}a\\
${newline}
;${linenum}d" ${config_path}
echo ""
fi
done
fi
done
done < "$config_path"
Using sed's s command to replace a line of that complexity is a losing proposition, because whatever delimiter you choose may appear in the line and mess things up. If these are in fact entire lines, it is better to delete them and insert a new one:
linenum=$(fgrep -nx -f "${line}" "${config_path}" | awk -F : "{print \$1}" )
[[ -n "${linenum}" ]] && sed -i "" "${linenum}a\\
${newline}
;${linenum}d" "${config_path}"
What this does is search for the line number of the line that matches $line in its entirety, then extracts the line number portion. fgrep is necessary otherwise the symbols in your file are interpreted as regular expressions. If there was a match, then it runs sed, appending the new line (a) and deleting the old one (d).
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 3 years ago.
For the below scripts, domains.txt is a text document that contains two lines:
google.com
adsfadsfad.net
I spent about two hours trying to troubleshoot why I couldn't get my grep within a if [[ ]] test to return what I was looking for:
#!/bin/bash
declare -a domain
while read domain; do
if [[ $(whois -H $domain | grep -i 'No match for "$domain"') ]]; then
echo "$domain is expired"
else
echo "exists"
fi
done < domains.txt
The above script would constantly indicate that both domains exist:
[root#localhost ~]# ./check-dns.sh
exists
exists
I confirmed that the following two tests worked perfectly fine from the shell:
#[root#localhost ~]# if [[ $(whois -H adsfadsfad.net | grep -i 'No match for "adsfadsfad.net"') ]]; then echo "true"; else echo "false"; fi
#true
#[root#localhost ~]# if [[ $(whois -H google.com | grep -i 'No match for "google.com"') ]]; then echo "true"; else echo "false"; fi
#false
I then confirmed that if I edited the script by removing the variable and replacing it with adsfadsfad.net, it produced the result I was looking for:
{snip}
if [[ $(whois -H $domain | grep -i 'No match for adsfadsfad.net') ]]; then
{snip}
adsfadsfad.net is expired
Finally, I settled on just running this test, which produces the correct results:
if [[ $(whois -H $domain | grep -i 'No match for') ]]; then
But my question is, why did the variable $domain break my script?
Because it was inside single quotes. That tells the shell not to interpolate the text inside, and thus the $domain remains literally dollar sign and the word domain. You need to remove it from the single quotes to get it to work:
if [[ $(whois -H "$domain" | grep -i 'No match for '"$domain" ) ]]
Or, in this case, because you don't need the single quotes at all:
if [[ $(whois -H "$domain" | grep -i "No match for $domain" ) ]]
You don't need [[ ... ]]. grep's exit status alone is sufficient for the if statement.
if whois -H $domain | grep -iq "No match for \"$domain\""; then
echo "$domain is expired"
else
echo "exists"
fi
Since you are testing grep's exit status, and not its output, you can suppress the output with the -q flag.
As already mentioned, bash does not evaluate the expression inside ', and you also want to preserve the double quotes. One thing you can do is to close the ' temporarily for the variable, then continue the text after it was evaluated:
#!/bin/bash
declare -a domain
while read domain; do
if [[ $(whois -H "$domain" | grep -i 'No match for domain "'$domain'"' ) ]]; then
echo "$domain is expired"
else
echo "exists"
fi
done < domains.txt
I also changed the text a bit so it matches to what the whois command prints.
I am trying to report lines found using grep and while.
I know you can use the following to compare a list of strings from inputs.txt and find them in your target file like so:
grep -f inputs.txt file_to_check
What I want is to read each line of the inputted strings and grep them individual in a loop.
So I have tried the following methods:
cat inputs.txt | while read line; do if grep "$line" filename_to_check; then echo "found"; else echo "not found"; fi; done
This returns nothing when I redirect the output to a file.
while read line
do
if grep "$line" file_to_check
then echo "found"
else
echo "not found"
fi
done < inputs.txt
Same as the first one but from what I found is better to do.
I know it iterates line by line because I can replace grep with echo $line and it prints each line; but either method doesn't return anything like grep -f above, instead it shows:
not found
not found
not found
.
. etc.
So what I'm looking for is something where it will iterate through each line and check it via grep using an if statement to determine if grep has actually found it or not. I know I may not have all proper logic but the output for what I want should look something like:
Found *matching line in file_to_check*
Found *matching line in file_to_check*
Not Found $line *(string that does not match)*
.
. etc.
You can also use && and || operators :
while read line; do
grep -q "$line" file_to_check && echo "$line found in file_to_check" || echo "$line not found in file_to_check"
done < inputfile > result.txt
The -q parameter of the grep just outputs a status code :
if $line is found, it outpouts 0 (True) the command after && will be evaluated
if not found, it outputs 1 (False) the command after || will evaluated
You can rewrite your final solution into
# Do not need this thanks to tr: file=$(dos2unix inputs.txt)
# Use -r so a line with backslashes will be showed like you want
while read -r line
do
# Not empty? Check with test -n
if [ -n "$(grep "${line}" filename)" ]; then
echo "found: ${line}"
else
echo "not found: ${line}"
fi
done < <(tr -d "\r" < "${file}")
Well, your if statement is pretty free form, you might need to clean it up a bit for bash to be able to read it. For example:
if [ "$(grep "$line" file_to_check)" != "" ]; then
echo "found: $line"
else
echo "not found: $line"
fi
This if statement will evaluate true if the grep command finds the line, because if it does it will spit the line out and will not be equal to "", or an empty string.
Here's my final solution:
file=$(dos2unix inputs.txt)
new_file=$file
while read line
do
if [ "$(grep "$line" filename)" != "" ]
then echo "found: $line"
else echo "not found: $line"
fi
done <$new_file
Thanks again!
abc.txt as
* This
Data1
Dat2
* Comment
data2
data3
When $line gets * as first char it does not work. It echoes garbage.
I tried various way but couldn't make it to work, how to check '*' in ksh varaible
while read line
do
fchar=`echo "$line" | cut -c 1-1`
#echo $fchar
if [[ ${char} = "*" ]]; then
continue
fi
if [[ ${char} = "#" ]]; then
continue
fi
echo ${line} >> stuff.txt
done<$abc.txt
Thanks.
Needs more quotes.
echo "$line" >>stuff.txt
...not
echo $line >>stuff.txt
...as the latter string-splits and glob-expands contents of the line, so it replaces * with a list of files in the current directory, replaces foo[bar] with foob if a file by that name exists, etc.
By the way, you'd be better off opening stuff.txt only once, not re-opening it every single time you want to add a new line. That'd look like this:
while read; do
fchar=${REPLY:0:1}
[[ "$fchar" = "*" ]] && continue
[[ "$fchar" = "#" ]] && continue
printf '%s\n' "$REPLY"
done >stuff.txt <abc.txt
By the way, you could also do this trivially with grep:
grep -E -v '^[*#]' <abc.txt >stuff.txt
Can anyone see whats wrong here? If I put X|9 in lan.db (or any db in this directory) and run the following code, the IF statement does not work. It's weird! if you echo $LINE, it is indeed pulling X|9 out of lan.db (or any db in this directory) and setting it equal to LINE, but it wont do the comparison.
DBREGEX="^[0-9]|[0-9]$"
shopt -s nullglob
DBARRAY=(databases/*)
i=0
for i in "${!DBARRAY[#]}"; do
cat ${DBARRAY[$i]} | grep -v \# | while read LINE; do
echo "$LINE" (Whats weird is that LINE DOES contain X|9)
if [[ !( $LINE =~ $DBREGEX ) ]]; then echo "FAIL"; fi
done
done
If however I just manually sent LINE="X|9" the same code (minus the while) works fine. ie LINE=X|9 fails, but LINE=9|9 succeeds.
DBREGEX="^[0-9]|[0-9]$"
Comment shopt -s nullglob
Comment DBARRAY=(databases/*)
Comment i=0
Comment for i in "${!DBARRAY[#]}"; do
Comment cat ${DBARRAY[$i]} | grep -v \# | while read LINE; do
LINE="X|9"
if [[ !( $LINE =~ $DBREGEX ) ]]; then echo "FAIL"; fi
Comment done
Comment done
* UPDATE *
UGH I GIVE UP
Now not even this is working...
DBREGEX="^[0-9]|[0-9]$"
LINE="X|9"
if [[ ! $LINE =~ $DBREGEX ]]; then echo "FAIL"; fi
* UPDATE *
Ok, so it looks like I have to escape |
DBREGEX="^[0-9]\|[0-9]$"
LINE="9|9"
echo "$LINE"
if [[ ! $LINE =~ $DBREGEX ]]; then echo "FAIL"; fi
This seems to work ok again
| has a special meaning in a regular expression. ^[0-9]|[0-9]$ means "starts with a digit, or ends with a digit". If you want to match a literal vertical bar, backslash it:
DBREGEX='^[0-9]\|[0-9]$'
for LINE in 'X|9' '9|9' ; do
echo "$LINE"
if [[ ! $LINE =~ $DBREGEX ]] ; then echo "FAIL" ; fi
done
You don't need round brackets in regex evaluation. You script is also creating a sub shell and making a useless use of cat which can be avoided.
Try this script instead:
while read LINE; do
echo "$LINE"
[[ "$LINE" =~ $DBREGEX ]] && echo "PASS" || echo "FAIL"
done < <(grep -v '#' databases/lan.db)