how to grep a variable in the shell program? [duplicate] - bash

This question already has answers here:
grep for expression containing variable
(2 answers)
Closed 9 years ago.
#!/bin/bash
for ((var=0; var<20; var++))
do
echo " Number is: $(grep 'Multiple_Frame = echo **$var**' 20mrf.txt | wc -l)" >>statisic.txt
done
This shell program cannot produce correct result which maybe the reason of wrong variable returning in the second grep command.
How can I grep a variable within the second echo sentence? to grep different things according to the var changing?
Many thanks!

As others have stated, the problem is that the single quotes prevent expansion of the variable. However, using $() allows you to use double quotes:
echo " Number is: $(grep "Multiple_Frame = echo **$var**" 20mrf.txt | wc -l)" >>statisic.txt
although I suspect something like this is what you meant:
echo " Number is: $(grep "Multiple_Frame = $var" 20mrf.txt | wc -l)" >>statisic.txt
You should also be aware that grep has an option to output the count so you can omit wc:
echo " Number is: $(grep -c "Multiple_Frame = $var" 20mrf.txt)" >>statisic.txt

#OP, doing what you do that way is rather inefficient. You are calling grep and wc 20 times on the same file. Open the file just once, and get all the things you want in 1 iteration of the file contents.
Example in bash 4.0
declare -A arr
while read -r line
do
case "$line" in
*"Multiple_Frame ="*)
line=${line#*Multiple_Frame = }
num=${line%% *}
if [ -z ${number_num[$num]} ] ;then
number_num[$num]=1
else
number_num[$num]=$(( number_num[$num]+1 ))
fi
;;
esac
done <"file"
for i in "${!number_num[#]}"
do
echo "Multiple_Frame = $i has ${number_num[$i]} counts"
done
similarly, you can use associative arrays in gawk to help you do this task.
gawk '/Multiple_Frame =/{
sub(/.*Multiple_Frame = /,"")
sub(/ .*/,"")
arr["Multiple_Frame = "$0]=arr["Multiple_Frame = "$0]+1
}END{
for(i in arr) print i,arr[i]
}' file

You have to store each substitution in a variable. Like this:
#!/bin/bash
for ((var=0; var < 20; var++))
do
count=`grep "Multiple_Frame = $var" 20mrf.txt | wc -l`
echo " Number is: $count" >> statisic.txt
done

Ok, the second [sic] problem is with your quoting on line 5. The reference to $var will never be expanded because it's contained within single quotes. You can fix that by replacing the single quotes (') with escaped double quotes (\").
The first [sic] problem is that you're trying to do too much in a single line, which causes your nesting problem with quotes. Break the line up into multiple commands, storing intermediary results as necessary. Yeah, it might run a tad slower, but you'll save a LOT of time debugging and maintaining it.
Trader's Second Law: If you have to choose between optimizing for performance, and optimizing for maintainability, ALWAYS choose to make your code more maintainable. Computers get faster all the time; Programmers don't.

The string inside $(...) is quoted with single quotes ('...') this quoting prevents the variable expansion. Use Double quotes instead. In your case:
#!/bin/bash
for ((var=0; var<20; var++))
do
echo " Number is: $(grep 'Multiple_Frame = echo **'"$var"'**' 20mrf.txt | wc -l)" >>statisic.txt
done
Note that in this example, it seems like the double quotes for the echo command are already opened, but you should note that the $(...) is evaluated first, and there is no double quotes inside it. So the change here is to close the single quote of the grep open double quote instead, and close the double quotes and reopen single quote later.
This lengthy explanation illustrates the benefit of breaking the expression apart, as suggested by other answers.

Related

script in bash two if condition and print in for loop

Hello im learning to script in bash, i trying to solve a little exercise.
The Exercise is this
If the variable named "basenew" contains the contents of the variable
named "valuebase". "basenew" must contain more than 113,469
characters. If both conditions are met, the script must then print
the last 20 characters of the variable "basenew".
My code is
#!/bin/bash
basenew="8dm7KsjU28B7v621Jls"
valuebase="ERmFRMVZ0U2paTlJYTkxDZz09Cg"
for i in {1..40}
do
basenew=$(echo $basenew | base64)
if [[ $basenew =~ $valuebase && ${#basenew} -ge 113469 ]] ; then
echo $i
echo $basenew | wc -c
StrLen=`echo ${basenew} | wc -c`
From=`expr $StrLen - 20`
echo $basenew | cut -c ${From}-${StrLen}
else
echo "error"
fi
done
But im stuck, because prints in the loop 28, and are the 20 last but isn't the correct answer.
Any advice to print the last 20 characters using tail -c 20?
Thanks
Why do you think the value of basenew will change during the execution of your loop? You never assign it. But you assign var that you never use, could it be the issue? Yes it is, you need to update basenew, not the unused var.
There is another problem with your script: cut applies your character selection on each line of the input. base64, by default, splits the output in lines of 76 characters. So, after a while, you have many short lines. And as they all are much shorter than your specification cut would only print a lot of newlines, not the last 20 characters of the string. You could change the behaviour of base64 with base64 -w0 but it would also change the results of each iteration and you would probably never match the reference string.
Instead of piping echo in cut follow joop's suggestion: "${basenew: -20: 20}" expands as the 20 last characters of the value of basenew. Or, a bit simpler but equivalent: "${basenew: -20}". Do not forget the space between : and -20, it is needed.
Finally, you could use more built-in bash constructs to avoid external commands (wc, echo, cut), use the modern $(...) instead of the old backticks, and use the arithmetic evaluation (((...))) instead of the obsolete expr. Try, maybe:
#!/usr/bin/env bash
# file: foo.sh
basenew="8dm7KsjU28B7v621Jls"
valuebase="ERmFRMVZ0U2paTlJYTkxDZz09Cg"
for i in {1..40}; do
basenew=$(echo "$basenew" | base64)
if [[ $basenew =~ $valuebase ]] && (( ${#basenew} >= 113469 )) ; then
printf '%d\n%d\n%s\n' "$i" "${#basenew}" "${basenew: -20}"
else
printf 'error\n'
fi
done
Demo:
$ ./foo.sh
error
...
error
28
113469
U2paTlJYTkxDZz09Cg==
error
...
error
Any advice to print the last 20 characters using tail
From the tail man-page:
-c, --bytes=[+]NUM
output the last NUM bytes; or use -c +NUM to output starting with
byte NUM of each file
Hence, if you have a variable x, you get the last 20 characters of it printed to stdout using
tail --bytes=20 <<<$x

Comparing SHA-512 Salted Hashes, displays filename during execution

Long story short during execution, i noticed that the "*" in the LIST somehow displays the name of the file i was executing. Is there anyway to let the code display "*" instead of displaying the filename ?
I am still a script newbie, could not think of any way. Please help ...
#!/bin/bash
LIST='W 2 v * %'
encr="Cd9AjUI4nGglIcP3MByrZUnu.hHBJc7.eR0o/v0A1gu0/6ztFfBxeJgKTzpgoCLptJS2NnliZLZjO40LUseED/"
salt="8899Uidd"
for i in $LIST
do
for j in $LIST
do
for k in $LIST
do
for l in $LIST
do
for a in $LIST
do
echo -n "$i$j$k$l$a "
test=`mkpasswd -m SHA-512 $i$j$k$l$a -s $salt`
if [ $test == $encr ] ; then
echo " Password is: $i$j$k$l$a"
exit
fi
done
done
done
done
done
#error displaying *
The same as echo * expands the globulation * to the filenames in the current directory, the same way a=*; for i in $a; do echo $i; done will print paths in the current directory. You can read more about quotes and escaping at various places in the net.
You can use bash arrays to correctly quotes elements:
LIST=(W 2 v "*" %)
for i in "${LIST[#]}"; do
Notes:
Don't use backticks `, they are discouraged. Use $(..) instead.
Quote your variables expansions [ "$test" = "$encr" ] to protect from common bugs like this.
The == is a bash extension, use = to be portable.
The method for generating permutations of elements looks strange and is not scalable. Consider writing a function or using other method. Even (ab-)using bash extension brace expansion echo {W,2,v,"*",%}{W,2,v,"*",%}{W,2,v,"*",%}{W,2,v,"*",%}{W,2,v,"*",%} looks shorter.

sed not parsing variable

I'm stuck with a bash script here...
I have variables:
hostname1="sxxx" hostname2="vbbbb" hostname3="sggg" hostname4="aaa" ...
I'm trying to change the 12th line of every files in a folder with the host-name variables.
The files are server1.txt server2.txt server3.txt server4.txt ...
I'm trying to do this with a while loop:
i=1
imax=1
while [[ $i -le 20 ]]
do
sed -i "12s/.*/$hostname$imax/" server$((imax)).txt
(( i++ ))
(( imax++ ))
if [[ imax -eq 21 ]]
then
imax=1
fi
done
what I want to do with sed is to concatenate the word host-name with imax and then use it as variable.
Maybe with this I'm clear enough:
$hostname=hostname$imax; //for exammple
sed -i "12s/.*/$hostname/" server$((imax)).txt // i need here the variable $hostname to have the content "sxxx"
You achieve this by using indirect parameter expansion,
${!var}
Here, var - is a dynamically generated variable name.
Try this:
sed -i "12s/.*/${!hostname}/" server$((imax)).txt
Example:
$ hostname1="sat"
$ i=1
$ hostval="hostname$i"
$ echo ${!hostval}
sat
I'd use the following. Use change, instead of switch in sed. Then strong quote my sed, and unquote my variables.
for i in {1..20}; do
eval h='$'hostname$i
sed -i '12c'$h'' server$i.txt
done
Bash 3 and over supports number ranges in a for looop, easier than your while loop. I also have no idea what you are doing with imax instead of just i, b/c you exit at 20, but change imax value to 1... which it will never use.
edit: b/c I misread
Basically, your problem comes from a variable interpretation.
Try using sed >=4.2.2, should work with your code.

case insensitive string comparison in bash

The following line removes the leading text before the variable $PRECEDING
temp2=${content#$PRECEDING}
But now i want the $PRECEDING to be case-insensitive. This works with sed's I flag. But i can't figure out the whole cmd.
No need to call out to sed or use shopt. The easiest and quickest way to do this (as long as you have Bash 4):
if [ "${var1,,}" = "${var2,,}" ]; then
echo "matched"
fi
All you're doing there is converting both strings to lowercase and comparing the results.
Here's a way to do it with sed:
temp2=$(sed -e "s/^.*$PRECEDING//I" <<< "$content")
Explanation:
^.*$PRECEDING: ^ means start of string, . means any character, .* means any character zero or more times. So together this means "match any pattern from start of string that is followed by (and including) string stored in $PRECEDING.
The I part means case-insensitive, the g part (if you use it) means "match all occurrences" instead of just the 1st.
The <<< notation is for herestrings, so you save an echo.
The only bash way I can think of is to check if there's a match (case-insensitively) and if yes, exclude the appropriate number of characters from the beginning of $content:
content=foo_bar_baz
PRECEDING=FOO
shopt -s nocasematch
[[ $content == ${PRECEDING}* ]] && temp2=${content:${#PRECEDING}}
echo $temp2
Outputs: _bar_baz
your examples have context-switching techniques.
better is (bash v4):
VAR1="HELLoWORLD"
VAR2="hellOwOrld"
if [[ "${VAR1^^}" = "${VAR2^^}" ]]; then
echo MATCH
fi
link: Converting string from uppercase to lowercase in Bash
If you don't have Bash 4, I find the easiest way is to first convert your string to lowercase using tr
VAR1=HelloWorld
VAR2=helloworld
VAR1_LOWER=$(echo "$VAR1" | tr '[:upper:]' '[:lower:]')
VAR2_LOWER=$(echo "$VAR2" | tr '[:upper:]' '[:lower:]')
if [ "$VAR1_LOWER" = "$VAR2_LOWER" ]; then
echo "Match"
else
echo "Invalid"
fi
This also makes it really easy to assign your output to variables by changing your echo to OUTPUT="Match" & OUTPUT="Invalid"

SED, using variables and in with an array

What I am trying to do is run the sed on multiple files in the directory Server_Upload, using variables:
AB${count}
Corresponds, to some variables I made that look like:
echo " AB1 = 2010-10-09Three "
echo " AB2 = 2009-3-09Foo "
echo " AB3 = Bar "
And these correspond to each line which contains a word in master.ta, that needs changing in all the text files in Server_Upload.
If you get what I mean... great, I have tried to explain it the best I can, but if you are still miffed I'll give it another go as I found it really hard to convey what I mean.
cd Server_Upload
for fl in *.UP; do
mv $fl $fl.old
done
count=1
saveIFS="$IFS"
IFS=$'\n'
array=($(<master.ta))
IFS="$saveIFS"
for i in "${array[#]}"
do
sed "s/$i/AB${count}/g" $fl.old > $fl
(( count++ ))
done
It runs, doesn't give me any errors, but it doesn't do what I want, so any ideas?
Your loop should look like this:
while read i
do
sed "s/$i/AB${count}/g" $fl.old > $fl
(( count ++ ))
done < master.ta
I don't see a reason to use an array or something similar. Does this work for you?
It's not exactly clear to me what you are trying to do, but I believe you want something like:
(untested)
do
eval repl=\$AB${count}
...
If you have a variable $AB3, and a variable $count, $AB${count} is the concatenation of $AB and $count (so if $AB is empty, it is the same as $count). You need to use eval to get the value of $AB3.
It looks like your sed command is dependent on $fl from inside the first for loop, even though the sed line is outside the for loop. If you're on a system where sed does in-place editing (the -i option), you might do:
count=1
while read i
do
sed -i'.old' -e "s/$i/AB${count}/g" Server_Upload/*.UP
(( count ++ ))
done < master.ta
(This is the entire script, which incorporates Patrick's answer, as well.) This should substitute the text ABn for every occurrence of the text of the nth line of master.ta in any *.UP file.
Does it help if you move the first done statement from where it is to after the second done?

Resources