I want to use awk in my bashscript, and this line clearly doesn't work:
line="foo bar"
echo $line | awk '{print $1}'
How do I escape $1, so it doesn't get replaced with the first argument of the script?
Your script (with single quotes around the awk script) will work as expected:
$ cat script-single
#!/bin/bash
line="foo bar"
echo $line | awk '{print $1}'
$ ./script-single test
foo
The following, however, will break (the script will output an empty line):
$ cat script-double
#!/bin/bash
line="foo bar"
echo $line | awk "{print $1}"
$ ./script-double test
​
Notice the double quotes around the awk program.
Because the double quotes expand the $1 variable, the awk command will get the script {print test}, which prints the contents of the awk variable test (which is empty). Here's a script that shows that:
$ cat script-var
#!/bin/bash
line="foo bar"
echo $line | awk -v test=baz "{print $1}"
$ ./script-var test
baz
Related reading: Bash Reference Manual - Quoting and Shell Expansions
As currently written, the $1 will not be replaced (since it's within single-quoted string, bash will not parse it)
If you write awk "{print $1}", bash will expand the $1 within the double-quoted string
Note that the variable expansion rules depend on the outermost level of quoting, so the $1 in "awk '{print $1}'" will be expanded
Related
I have this script that's designed to assign variables to commands that collect information about a system and then echo them back. This works very well for the first few commands, but the last one continues to return the value without "PRETTY_NAME=" stripped out of the output.
Is there some problem with this that I'm not seeing?
I have tried using grep to separate awk:
grep PRETTY_NAME /etc/*-release | awk -F '=' '{print $2}'
Using escaped quotes:
awk -F \"=\" '/PRETTY_NAME/ {print $2}' /etc/*-release
Whole block (edited somewhat for relevance)
declare -A CMDS=(
[primaryMacAddress]="cat /sys/class/net/$(ip route show default | awk '/default/ {print $5}')/address"
[primaryIpAddress]="hostname --ip-address"
[hostname]="hostname"
[osType]="awk -F '=' '/PRETTY_NAME/ {print $2}' /etc/*-release"
)
#This bit is actually nested in another function
for kpair in "${!CMDS[#]}" do
echo "$kpair=\"$( eval ${CMDS[$kpair]} )\""
done
Results when run from .sh file:
osType="PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 (Maipo)""
expected:
osType=""Red Hat Enterprise Linux Server 7.4 (Maipo)""
When this command is run by itself, it seems to work as intended:
$ awk -F '=' '/PRETTY_NAME/ {print $2}' /etc/*-release
"Red Hat Enterprise Linux Server 7.4 (Maipo)"
Because your Awk command is specified in double quotes, interior dollar signs are subject to special treatment: the $2 is treated as a parameter substitution by your shell, and so the array element doesn't store the text $2 but rather its expansion. The Awk interpreter never sees the $2 syntax.
However, you have a second problem in your command dispatcher. Your eval command does not prevent word splitting:
eval ${CMDS[$kpair]}
you want this:
eval "${CMDS[$kpair]}"
without the quotes, your command is arbitrarily chopped into fields on whitespace. Then eval catenates the pieces together, using one space between them, and evaluates the resulting syntax. The difference can be demonstrated with the following example:
$ cmd="awk '/foo/ { print \$1\" \"\$2 }'"
$ echo 'foo a' | eval $cmd
foo a
$ echo 'foo a' | eval "$cmd"
foo a
We can just use echo to understand the issue:
$ echo $cmd
awk '/foo/ { print $1" "$2 }'
$ echo "$cmd"
awk '/foo/ { print $1" "$2 }'
The substitution of $cmd and the subsequent word splitting is done irrespective of any shell syntax that `cmd contains. We can see the pieces like this:
$ for x in $cmd ; do echo "<$x>" ; done
<awk>
<'/foo/>
<{>
<print>
<$1">
<"$2>
<}'>
When we execute eval $cmd, the above pieces are generated and re-combined by eval and evaluated. Needless to say, you don't want your command syntax to be chopped up and re-combined like this; who knows what sort of hidden bug will arise. It may be okay for the commands you have now, but as a generic command dispatch mechanism, it is flawed.
I have found that the command expanded from variable won't run the latter parts in a pipe. For example, as following test.sh:
#!/bin/bash
y='echo hello man | awk "{print \$1}"'
$y
eval $y
y='echo hello'
$y
The output of the script is:
hello man | awk "{print \$1}"
hello
hello
The first $y only execute echo hello man but not execute awk "{print $1}" of the pipe. That's why?
My bash version is 4.3.48.
That's because the variable is expanded after the pipes and redirection is already done. So in that case | is just another argument to echo, not a pipe that's interpreted by the shell.
Recommended reading: http://mywiki.wooledge.org/BashFAQ/050
When executing the command
echo hello man | awk '{print $1}'
the shell will see the | and setup a pipeline, on one side it will run the command echo hello man and on the other awk '{print $1}'. The commands then get subjected to word splitting so you run the command echo with 2 arguments: hello and man. On the other side you run the command awk with a single argument (because of the quoting) "{print \$1}"
When that command is stored as a string though the shell first looks at the command $y and sees no redirection to do. It then expands $y and then does word splitting on it. It gets expanded to the same looking string, but now it's too late for redirection. So it gets split into the words echo, hello, man, |, awk, "{print, \$1}" (note that the argument to awk now gets split because the quotes inside the string are part of the string, not syntactical)
The first word in that list is echo so that's the command, and all the rest of the words are passed as arguments to it, which is why you see the output
hello man | awk "{print \$1}"
When you do the eval line, it takes that same string and tells bash to parse it as though it had been typed so the pipe once again becomes syntatical and causes the pipeline.
Because echo puts its arguments on the same line it's a little tougher sometimes to see what's happening, if we replace it with printf '%s\n' each argument will become its own line though:
$ y='printf %s\n hello man | awk "{print \$1}"'
$ $y
hello
man
|
awk
"{print
\$1}"
Here is my code:
!/bin/bash
#Basic Syntax - pfc.sh [filename] [keyword] [delimiter]
cat $1 |awk -F $2 '{print $2}' |awk -F $3 '{print$1}'
This works as is, however the 2nd and 3rd variables need to be surrounded by single ticks to work and I would not like for this to be necessary. Thanks in advance.
How do I add single ticks to my variables so my team doesn't have to.
Example of how it currently works:
sh pfc.sh variable1 'variable2' 'variable3'
Sorry, there is no work around for that. Single quotes(not ticks) tells the shell to interpret the pattern literally. For example consider that you have 'my argument',with out enclosing the argument inside single quotes, the shell interprets it as two arguments - my and argument.
For the same reason, your script too is flawed. It should have been
cat "$1" |awk -F "$2" '{print $2}' |awk -F "$3" '{print $1}'
# mind the double quotes which prevents word splitting
# And you can't replace double quotes with single quotes as
# variable expansion will not happen inside single quotes.
I would like to replace a variable inside the the awk command with a bash variable.
For example:
var="one two three"
echo $var | awk "{print $2}"
I want to replace the $2 with the var variable. I have tried awk -v as well as something like awk "{ print ${$wordnum} } to no avail.
Sightly different approach:
$ echo $var
one two three
$ field=3
$ echo $var | awk -v f="$field" '{print $f}'
three
$ field=2
$ echo $var | awk -v f="$field" '{print $f}'
two
You've almost got it...
$ myfield='$3'
$ echo $var | awk "{print $myfield}"
three
The hard quotes on the first line prevent interpretation of $3 by the shell. The soft quotes on the second line allow variable replacement.
You can concatenate parts of awk statements with variables. Maybe this is what you want in your script file:
echo $1|awk '{print($'$2');}'
Here the parts {print($ and the value of local variable $2 and );} are concatenated and given to awk.
EDIT: After some advice rather don't use this. Maybe as a one-time solution. It's better to get accustomed to doing it right right away - see link in first comment.
I'm trying to do some manipulation with Wordpress and I'm trying to write a script for it...
# cat /usr/local/uftwf/_wr.sh
#!/bin/sh
# $Id$
#
table_prefix=`grep ^\$table_prefix wp-config.php | awk -F\' '{print $2}'`
echo $table_prefix
#
Yet I'm getting following output
# /usr/local/uftwf/_wr.sh
ABSPATH ABSPATH wp-settings.php_KEY LOGGED_IN_KEY NONCE_KEY AUTH_SALT SECURE_AUTH_SALT LOGGED_IN_SALT NONCE_SALT wp_0zw2h5_ de_DE WPLANG WP_DEBUG s all, stop editing! Happy blogging. */
#
Running from command line, I get the correct output that I'm looking for:
# grep ^\$table_prefix wp-config.php | awk -F\' '{print $2}'
wp_0zw2h5_
#
What is going wrong in the script?
The problem is the grep command:
table_prefix=`grep ^\$table_prefix wp-config.php | awk -F\' '{print $2}'`
It either needs three backslashes - not one - or you need to use single quotes (which is much simpler):
table_prefix=$(grep '^$table_prefix' wp-config.php | awk -F"'" '{print $2}')
It's also worth using the $( ... ) notation in general.
The trouble is that the backquotes removes the backslash, so the shell variable is evaluated, and what's passed to grep is, most likely, just ^, and each line starts with a beginning of line.
This has all the appearance as though the grep is not omitting all the lines that are not matching, when you issue the echo $table_prefix without quotes it collapses all the white space into a single output line, if you issue an: echo "$table_prefix", you would see the match with all the other white-space that was output.
I'd recommend the following sed expression instead:
table_prefix=$(sed -n "s/^\$table_prefix.*'\([^']*\)'.*/\1/p" wp-config.php)
You should try
#!/bin/sh
table_prefix=$(awk -F"'" '/^\$table_prefix/{print $2}' wp-config.php)
echo $table_prefix
Does this one work for you?
awk -F\' '/^\$table_prefix/ {print $2}' wp-config.php
Update
If you are using shell scripting, there is no need to call up awk, grep:
#!/bin/sh
while read varName op varValue theRest
do
if [ "_$varName" = "_\$table_prefix" ]
then
table_prefix=${varValue//\'/} # Remove the single quotes
table_prefix=${table_prefix/;/} # Remove the semicolon
break
fi
done < wp-config.php
echo "Found: $table_prefix"