I'm trying to use a variable in a grep regex. I'll just post an example of the failure and maybe someone can suggest how to make the variable be evaluated while running the grep command. I've tried ${var} as well.
$ string="test this"
$ var="test"
$ echo $string | grep '^$var'
$
Since my regex should match lines which start with "test", it should print the line echoed thru it.
$ echo $string
test this
$
You need to use double quotes. Single quotes prevent the shell variable from being interpolated by the shell. You use single quotes to prevent the shell from doing interpolation which you may have to do if your regular expression used $ as part of the pattern. You can also use a backslash to quote a $ if you're using double quotes.
Also, you may need to put your variable in curly braces ${var} in order to help separate it from the rest of the pattern.
Example:
$ string="test this"
$ var="test"
$ echo $string | grep "^${var}"
Related
I am trying to find and replace a specific text content using the sed command and to run it via a shell script.
Below is the sample script that I am using:
fp=/asd/filename.txt
fd="sed -i -E 's ($2).* $2:$3 g' ${fp}"
eval $fd
and executing the same by passing the arguments:
./test.sh update asd asdfgh
But if the argument string contains $ , it breaks the commands and it is replacing with wrong values, like
./test.sh update asd $apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1.
How can I make sure that the values inside the variables are not expanded because of the $?
Updated
sh file test.sh
set -xv
fp="/asd/filename.txt"
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
text file filename.txt
hello:world
Outputs
1)
./test.sh update hello WORLD
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
++ sed -iE 's/(hello).*/hello:WORLD/g' /asd/filename.txt
2)
./test.sh update hello '$apr1$hosgaxyv$D0KXp5dCyZ2BUYCS9BmHu1'
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
++ sed -iE 's/(hello).*/hello:'\''$'\''apr1'\''$'\''hosgaxyv'\''$'\''D0KXp5dCyZ2BUYCS9BmHu1/g' /asd/filename.txt
In both the case , its not replacing the content
You don't need eval here at all:
fp=/asd/filename.txt
sed -i -E "s/(${2//'$'/'\$'}).*/\1:${3//'$'/'\$'}/g" "$fp"
The whole sed command is in double quotes so variables can expand.
I've replaced the blank as the s separator with / (doesn't really matter in the example).
I've used \1 to reference the first capture group instead of repeating the variable in the substitution.
Most importantly, I've used ${2//'$'/'\$'} instead of $2 (and similar for $3). This escapes every $ sign as \$; this is required because of the double quoting, or the $ get eaten by the shell before sed gets to see them.
When you call your script, you must escape any $ in the input, or the shell tries to expand them as variable names:
./test.sh update asd '$apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1.'
Put the command-line arguments that are filenames in single quotes:
./test.sh update 'asd' '$apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1'
must protect all the script arguments with quotes if having space and special shell char, and escape it if it's a dollar $, and -Ei instead of -iE even better drop it first for test, may add it later if being really sure
I admit i won't understant your regex so let's just get in the gist of solution, no need eval;
fp=/asd/filename.txt
sed -Ei "s/($2).*/$2:$3/g" $fp
./test.sh update asd '\$apr1\$HnIF6bOt\$9m3NzAwr.aG1Yp.t.bpIS1.'
I have a variable containing a string with a quote:
echo $variable
It's my variable
to be able to use this variable as a legend for ffmpeg, I need to add 5 backslashes before the quote:
variable="It\\\\\'s my variable"
I'm confused as to what syntax I should use, as the backslashes and quotes have very specific meanings in bash replace commands. I have tried this:
variable=`echo $variable | tr "'" "\\\\\'"`
but it does not produce the correct result
You can just use some single quotes yourself to tell bash not to interpret those slashes:
variable="It"'\\\\\'"'s my variable"
Edit: To convert an existing variable use:
variable=${variable//\'/'\\\\\'\'}
This works fine, is portable and does not have the problems of sed:
echo "$v" |perl -pe "s/'/\x5c\x5c\x5c\x5c\x5c'/g"
PS: \x5c is the ascii code of slash \
I am writing a BASH script and I am using the bash command. Which one of the following is correct (or are both incorrect)?
bash $pbs_dir/${module_name}.${target_ID}.${instance_ID}.pbs
or
bash \$pbs_dir/\${module_name}.\${target_ID}.\${instance_ID}.pbs
\$ will be expanded to literal $, so there is a big difference:
$ a="hello"
$ echo $a
hello
$ echo \$a
$a
Also note that you almost always want to double quote your parameter expansions to avoid word splitting and pathname expansion:
echo "$a"
So you properly want to use the following:
bash "$pbs_dir/${module_name}.${target_ID}.${instance_ID}.pbs"
I am trying to escape backslash in AWK. This is a sample of what I am trying to do.
Say, I have a variable
$echo $a
hi
The following works
$echo $a | awk '{printf("\\\"%s\"",$1)'}
\"hi"
But, when I am trying to save the output of the same command to a variable using command substitution, I get the following error:
$ q=`echo $a | awk '{printf("\\\"%s\"",$1)'}`
awk: {printf("\\"%s\"",$1)}
awk: ^ backslash not last character on line
I am unable to understand why command substitution is breaking the AWK. Thanks a lot for your help.
Try this:
q=$(echo $a | awk '{printf("\\\"%s\"",$1)}')
Test:
$ a=hi
$ echo $a
hi
$ q=$(echo $a | awk '{printf("\\\"%s\"",$1)}')
$ echo $q
\"hi"
Update:
It will, it just gets a littler messier.
q=`echo $a | awk '{printf("\\\\\"%s\"",$1)}'`
Test:
$ b=hello
$ echo $b
hello
$ t=`echo $b | awk '{printf("\\\\\"%s\"",$1)}'`
$ echo $t
\"hello"
Reference
Quoting inside backquoted commands is somewhat complicated, mainy
because the same token is used to start and to end a backquoted
command. As a consequence, to nest backquoted commands, the backquotes
of the inner one have to be escaped using backslashes. Furthermore,
backslashes can be used to quote other backslashes and dollar signs
(the latter are in fact redundant). If the backquoted command is
contained within double quotes, a backslash can also be used to quote a
double quote. All these backslashes are removed when the shell reads
the backquoted command. All other backslashes are left intact.
The new $(...) avoids these troubles.
Don't get into bad habits with backticks, quoting and parsing shell variables to awk The correct way to do this is:
$ shell_var="hi"
$ awk -v awk_var="$shell_var" -v c='\' 'BEGIN{printf "%s%s\n",c,awk_var}'
\hi
$ res=$(awk -v awk_var="$shell_var" -v c='\' 'BEGIN{printf "%s%s\n",c,awk_var}')
$ echo "$res"
\hi
I have a string, "$server['fish_stick']" (disregard double quotes)
I don't know how to successfully grep for an exact match for this string. I've tried many ways.
I've tried,
rgrep -i \$'server'\[\''fish'\_'stick'\'\] .
rgrep -i "\$server\[\'fish\_stick\'\]" .
rgrep -i '\$server\[\'fish\_stick\'\]' .
Is it single quotes that are causing my issue?
When I echo the first grep out it shows exactly what I want to search but returns garbage results like anything with $server in it.
Please help and explain, thank you!
The main problem here is that you are not quoting the argument being passed to grep. The only thing that needs to be escaped is \$ (if double quoted) and []. If you want the exact string (not using regex), just use fgrep (grep -F) which does exact string matching:
grep -F "\$server['fish_stick']"
Works on my system:
$ foo="\$server['fish_stick']"
$ echo "$foo" | grep -F "\$server['fish_stick']"
$server['fish_stick']
Using regex:
$ echo "$foo" | grep "\$server\['fish_stick'\]"
$server['fish_stick']
Using regex and handling nested single quotes:
$ echo "$foo" | grep '\$server\['\''fish_stick'\''\]'
$server['fish_stick']
Inside of single quotes, nested single quotes can not be not be escaped. You have to close the quotes, and then reopen it to "escape" the single quotes.
http://mywiki.wooledge.org/Quotes
I don't suppose you're asking how to get that string into a variable without having quoting issues. If you are, here's a way using a here-document:
str=$(cat <<'END'
$foo['bar']
END
)
To address your concern about escaping special characters for grep, you could use sed to put a backslash before any non-alphanumeric character:
grep "$(sed 's/[^[:alnum:]]/\\&/g' <<< "$str")" ...
When used with set -x, the grep command looks like: grep '\$foo\[\'\''bar\'\''\]' ...