I am trying to read a parameter file in a shell script and would want to skip the lines which start with "#". Have been trying it on Ubuntu VM (default bash) and for something that I can't understand, it doesn't seem to work.
Following is the pseudo-code that I am using:
while read line
do
if [ grep -q "#" <<< "$line" ]; then
## Do nothing (Commented Out)
echo "$line Line is Commented out"
elif [ "$line" = "" ]; then
## Do nothing (Blank Line)
echo "Blank line"
else
#echo "read line is $line"
...some logic here
fi
done <input_file.ini
This yields the the following error: Syntax error: redirection unexpected
The if [[ $line == *#* ]] construct doesn't seem to work. My earlier experience was on AIX where everything worked fine.
Could someone guide me what I am doing wrong here?
PS: On a related note, how do I handle cases where I don't want to do anything? e.g. when there is no '#' character in the read line, I don't want to do anything. I can't leave my if block blank so I am just using echo 'some random' text. My task works good but just wanted to understand what's a good practice to handle this.
Your code is clearly running with /bin/sh, not bash.
An alternative to [[ $line = *"#"* ]] that works with /bin/sh is case.
Thus, the following will work with /bin/sh, or when invoked with sh yourscript:
#!/bin/sh
while read -r line; do : line="$line"
case $line in
*"#"*) echo "Line is commented out: $line";;
"") echo "Line is empty" ;;
*) key=${line%%=*}
value=${line#*=}
eval "$key="'$line' # unsafe, but works with /bin/sh, which doesn't have better
# indirect assignment approaches.
printf '%s\t\t-\t\t%s\n' "$key" "$value"
;;
esac
done <input_file.ini
Alternately, consider putting in a guard to handle the case when your script is invoked with a non-bash shell:
#!/bin/bash
case $BASH_VERSION in
'')
echo "ERROR: Run with a non-bash shell" >&2
if [ "$tried_reexec" ]; then
echo "ERROR: Already attempted reexec and failed" >&2
exit 1
fi
if [ -s "$0" ]; then
export tried_reexec=1
exec bash "$0" "$#"
fi
;;
esac
while read -r line; do
if [[ $line = *"#"* ]]; then
echo "Line is Commented out: $line"
elif [[ "$line" = "" ]]; then
echo "Blank line"
else
key=${line%%=*}; value=${line#*=}
printf -v "$key" %s "$value"
printf '%s\t\t-\t\t%s\n' "$key" "$value"
fi
done <input_file.ini
I really wasn't able to figure out the exact problem with double [[ ]] and the character search in a string. Thanks to everyone who tried to help me out. However this was acting as a deterrent and I didn't want to continue to fiddle for too long, I used a slightly different approach to handle my situation.
The following code works for me now:
while read line
do
first_char=`echo $line | cut -c 1`
if [ "$first_char" = "#" ]; then
: "do nothing here. Line is commented out"
elif [ "$line" = "" ]; then
: "do nothing here. Blank line"
else
KEY="$(echo $line | cut -d '=' -f1)"
VALUE="$(echo $line | cut -d '=' -f2)"
printf \v "$KEY" %s "$VALUE"
echo "$KEY\t\t-\t\t$VALUE"
fi
done < ${SCHEDULER_LOC}/inputs/script_params.ini
Also I was able to learn few things so incorporated them as well. I did get few negative scores for this question. Understandably so since this might be rudimentary for the experts but it was a genuine problem I was seeking some guidance on. Still, I am thankful that I learnt something new. Kudos to the community.
Related
#!/bin/bash
set -e
deb_folder='/home'
myinstall(){
deb=$1
temp="${1%.*}"
num="${temp##*.}"
temp2="${temp%.*}"
method="${temp2##*.}"
case "$method" in
md5)
md5=md5sum $deb
echo 'here'
if [[ "${md5:0:3}${md5: -3}" == "$num" ]]; then echo 'correct' else echo $deb'md5 error';false;fi
;;
256)
sha256=sha256sum $deb
if [[ "${sha256:0:3}${sha256: -3}" == "$num" ]]; then apt-get install $deb; else echo $deb'sha256 error';false;fi
;;
*) echo $deb'sum type wrong'
;;
esac
}
myinstall "${deb_folder}/rstudio-1.4.1106-amd64.md5.e596d3.deb"
Expect result of above bash script is correct or /home/rstudio-1.4.1106-amd64.md5.e596d3.debmd5 error,but I got here after change md5=md5sum $deb to md5=$(md5sum $deb).
Where is the problem?
Problem 1
Instead of md5=md5sum $deb you probably meant md5=$(md5sum $deb) or even better md5=$(md5sum "$deb"). The same goes for sha256=sha256sum $deb.
md5=$(md5sum $deb) runs the command md5sum $deb and stores its output in the variable md5.
md5=md5sum $deb runs the "command" $deb while setting the environment variable md5=md5sum for this command. You may have seen this construct in idioms like IFS= read -r line or LC_ALL=C sort before.
Problem 2
The following if has only one branch. That else is very misleading.
if [[ "${md5:0:3}${md5: -3}" == "$num" ]]; then echo 'correct' else echo $deb'md5 error';false;fi
If written properly formatted, the problem becomes clear:
if [[ "${md5:0:3}${md5: -3}" == "$num" ]]; then
echo 'correct' else echo $deb'md5 error'
false
fi
Here the else is not a keyword, but a simple argument to echo. If you enter the if you would get the output correct else echo /home/rstudio-1.4.1106-amd64.md5.e596d3.debmd5 error.
To fix this, add a ; or linebreak before else.
You may as well fix the check "${md5:0:3}${md5: -3}" == "$num". I don't think these things will ever be equal. Execute your script with set -x to print the values of your variables, then you see the problems.
I am very new to Bash Scripting and I have a question regarding my CheckOurCodingRules.sh script:
I want to search for every 'hPar,' in a textfile and if found it should be checked if there is a also a 'const' in the same row.
Thats what I got so far but there is something wrong here:
while read line
do
if [[ $line == *hPar\,* ]] && [[ $line == *const\*]];then
DOCUMENTATION_TEST_A=1
else
echo DOCUMENTATION_TEST_A=0
fi
done < $INPUT_FILE
if [[DOCUMENTATION_TEST_A=0]];then
echo "error: Rule1: No const before hpar"
fi
There are a couple of issues with your script, see the code below which works for me:
DOCUMENTATION_TEST_A=0 # initial value
while read line
do
# spaces between conditional and brackets, no backslashes
if [[ $line == *hPar,* ]] && [[ $line == *const* ]]
then
DOCUMENTATION_TEST_A=1
break # optional, no need to scan the rest of the file
fi
done < $INPUT_FILE
# spaces and $, -eq is used for numerical comparisons
if [[ $DOCUMENTATION_TEST_A -eq 0 ]];
then
echo "error: Rule1: No const before hpar"
fi
A cleaner solution would be to use grep:
if ! grep "hPar," $INPUT_FILE | grep "const" >/dev/null
then
echo "error: Rule1: No const before hpar"
fi
This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 6 years ago.
I'm trying to loop through allURLs.txt and check if every entry in that file exists in PDFtoCheck.pdf. I know of a tool called pdfgrep, but can't seem to apply it to suit my objective.
#!/bin/bash
entriesMissing=0;
cat allURLs.txt | while read line
do
# do something with $line here
if [ ! -z echo `pdfgrep "$line" PDFtoCheck.pdf` ];
then
echo "yay $line";
else
echo "$line not found";
entriesMissing=$[$entriesMissing+1];
fi
done
echo "DONE";
echo "There are $entriesMissing entries missing!";
Despite placing dummy values in allURLs.txt, entires which are present in allURLs.txt but not in PDFtoCheck.pdf are not reflected in the output. Any idea how to make it work as intended?
Please note that a subshell is created when piping: cat file | while. You should use file redirection instead: while ... do; done < file.
As far as I can see pdfgrep supports the -q quiet flag, so you can just use it in the if-statement.
entriesMissing=0
while IFS= read -r line; do
if pdfgrep -q -- "$line" PDFtoCheck.pdf; then
printf "Found '%s'\n" "$line"
else
printf "'%s' not found\n" "$line"
((entriesMissing++))
fi
done < allURLs.txt
printf "There are %d entries missing\n" "%entriesMissing"
I also changed the increment to ((... ++))
Extending my comment as answer. I'm using -c option which is also available in pdfgrep :
entriesMissing=0
while read line
do
# do something with $line here
if [ $(grep -c "$line" b) -eq 0 ]
then
((entriesMissing++))
echo "$line not found"
else
echo "yay $line"
fi
done < allURLs.txt
echo "DONE"
echo "There are $entriesMissing entries missing!";
One thing I want point out in your code that you are incrementing entriesMissing inside a subshell(pipe) which doesn't get reflected at the last line. Hope it helps.
I am trying to report lines found using grep and while.
I know you can use the following to compare a list of strings from inputs.txt and find them in your target file like so:
grep -f inputs.txt file_to_check
What I want is to read each line of the inputted strings and grep them individual in a loop.
So I have tried the following methods:
cat inputs.txt | while read line; do if grep "$line" filename_to_check; then echo "found"; else echo "not found"; fi; done
This returns nothing when I redirect the output to a file.
while read line
do
if grep "$line" file_to_check
then echo "found"
else
echo "not found"
fi
done < inputs.txt
Same as the first one but from what I found is better to do.
I know it iterates line by line because I can replace grep with echo $line and it prints each line; but either method doesn't return anything like grep -f above, instead it shows:
not found
not found
not found
.
. etc.
So what I'm looking for is something where it will iterate through each line and check it via grep using an if statement to determine if grep has actually found it or not. I know I may not have all proper logic but the output for what I want should look something like:
Found *matching line in file_to_check*
Found *matching line in file_to_check*
Not Found $line *(string that does not match)*
.
. etc.
You can also use && and || operators :
while read line; do
grep -q "$line" file_to_check && echo "$line found in file_to_check" || echo "$line not found in file_to_check"
done < inputfile > result.txt
The -q parameter of the grep just outputs a status code :
if $line is found, it outpouts 0 (True) the command after && will be evaluated
if not found, it outputs 1 (False) the command after || will evaluated
You can rewrite your final solution into
# Do not need this thanks to tr: file=$(dos2unix inputs.txt)
# Use -r so a line with backslashes will be showed like you want
while read -r line
do
# Not empty? Check with test -n
if [ -n "$(grep "${line}" filename)" ]; then
echo "found: ${line}"
else
echo "not found: ${line}"
fi
done < <(tr -d "\r" < "${file}")
Well, your if statement is pretty free form, you might need to clean it up a bit for bash to be able to read it. For example:
if [ "$(grep "$line" file_to_check)" != "" ]; then
echo "found: $line"
else
echo "not found: $line"
fi
This if statement will evaluate true if the grep command finds the line, because if it does it will spit the line out and will not be equal to "", or an empty string.
Here's my final solution:
file=$(dos2unix inputs.txt)
new_file=$file
while read line
do
if [ "$(grep "$line" filename)" != "" ]
then echo "found: $line"
else echo "not found: $line"
fi
done <$new_file
Thanks again!
Can anyone see whats wrong here? If I put X|9 in lan.db (or any db in this directory) and run the following code, the IF statement does not work. It's weird! if you echo $LINE, it is indeed pulling X|9 out of lan.db (or any db in this directory) and setting it equal to LINE, but it wont do the comparison.
DBREGEX="^[0-9]|[0-9]$"
shopt -s nullglob
DBARRAY=(databases/*)
i=0
for i in "${!DBARRAY[#]}"; do
cat ${DBARRAY[$i]} | grep -v \# | while read LINE; do
echo "$LINE" (Whats weird is that LINE DOES contain X|9)
if [[ !( $LINE =~ $DBREGEX ) ]]; then echo "FAIL"; fi
done
done
If however I just manually sent LINE="X|9" the same code (minus the while) works fine. ie LINE=X|9 fails, but LINE=9|9 succeeds.
DBREGEX="^[0-9]|[0-9]$"
Comment shopt -s nullglob
Comment DBARRAY=(databases/*)
Comment i=0
Comment for i in "${!DBARRAY[#]}"; do
Comment cat ${DBARRAY[$i]} | grep -v \# | while read LINE; do
LINE="X|9"
if [[ !( $LINE =~ $DBREGEX ) ]]; then echo "FAIL"; fi
Comment done
Comment done
* UPDATE *
UGH I GIVE UP
Now not even this is working...
DBREGEX="^[0-9]|[0-9]$"
LINE="X|9"
if [[ ! $LINE =~ $DBREGEX ]]; then echo "FAIL"; fi
* UPDATE *
Ok, so it looks like I have to escape |
DBREGEX="^[0-9]\|[0-9]$"
LINE="9|9"
echo "$LINE"
if [[ ! $LINE =~ $DBREGEX ]]; then echo "FAIL"; fi
This seems to work ok again
| has a special meaning in a regular expression. ^[0-9]|[0-9]$ means "starts with a digit, or ends with a digit". If you want to match a literal vertical bar, backslash it:
DBREGEX='^[0-9]\|[0-9]$'
for LINE in 'X|9' '9|9' ; do
echo "$LINE"
if [[ ! $LINE =~ $DBREGEX ]] ; then echo "FAIL" ; fi
done
You don't need round brackets in regex evaluation. You script is also creating a sub shell and making a useless use of cat which can be avoided.
Try this script instead:
while read LINE; do
echo "$LINE"
[[ "$LINE" =~ $DBREGEX ]] && echo "PASS" || echo "FAIL"
done < <(grep -v '#' databases/lan.db)