How to use awk variable in search?
Name="jony"
awk -v name="$Name" '/name/ {print $0}' file
this will search for string name, not for $Name which is actually jony.
Correct, awk won't recogize variables in / /. You can do:
Name="jony"
awk -v name="$Name" '$0 ~ name' file
Since print is awk's default behavior we can avoid using it here.
Hope I understood problem correctly:
Why wont you try following one:
awk '/'"$Name"'/ { print } ' testfile
When writing an AWK one-liner, you could quote the script with either the single quotes or double quotes. In the latter case the shell does all the substitution directly so that you do not need to pass the variable into the script via -v option:
Name="jony"
awk "/$Name/" file
# this works. after shell has performed substitutions, the line looks like
awk "/jony/" file
[bad!] Or even without quotes if the name does not contain spaces:
awk /$Name/ file
All the simplicity vanishes as soon as you want to use $ in the script, including awk special variables that use $0, $1, etc, because you will have to escape the dollar sign to prevent shell variable expansion.
awk "/$Name/ {print \$0}"
In addition you will have to escape the double quotes to add literal text to the script. Looks clumsy:
awk "/$Name/ {print \"Found in: \" \$0}"
To crown it all, negating regular expression with double quotes will cause a shell error:
awk "!/$Name/"
#error> ... event not found ...
The error will happen if $Name itself contains ! sign. This makes using double quotes unreliable.
So, to be on the safe side, prefer single quotes :)
Related
Does awk support using a different delimiter character than ' in print? e.g. Instead of awk '{print $1}', something like awk -d # #{print $1}#
I was actually looking at the C source code and it's a pretty short program; is there an alternative version that allow that?
It can't: The ' isn't passed to awk; instead, it's understood by the shell itself. Thus, when you run awk '{print $1}', what you're actually calling at the OS level is something like:
/* this is C syntax, so the double-quotes are C quotes; only their contents are literal */
execvp("awk", { "awk", "{print $1}", NUL });
Notably, the single-quotes aren't there at all any more -- they were parsed out by the shell when it understood them as instructions for how it should break the command into an argument list.
To see it in another way, consider if you put your script in a separate file and called
awk -f my.awk
The contents of my.awk would simply be
{print $1}
not
'{print $1}'
The single quotes are only used by the shell to ensure that the script is passed literally to awk, rather than being subject to any particular shell processing that could change the script before awk could read it.
I am learning Bash scripts.
my data.txt contains:
A Orange
B Apple
I have tried
echo "enter"
read fruit
awk -F'[: ]' '$2 == "Apple"' data.txt
a=`awk -F'[: ]' '\$2 == "$fruit"' data.txt` # returns null
echo "$a"
Why isn't my awk command is not working when I am using my variable 'a' with it?
The crux of the problem is that you're trying to use a single-quoted string as if it were a double-quoted one.
What you meant to do when you used:
'\$2 == "$fruit"' # DOES NOT WORK - nothing is escaped or expanded
was:
"\$2 == \"$fruit\"" # keep literal $2, expand $fruit
Anything inside a single-quoted shell string is taken literally, so variable references aren't expanded, and nothing needs escaping - nor indeed can be escaped (the \ would become a literal part of the string too).
That said, it's better to keep the worlds of the shell and the Awk script separate, so as to prevent confusion, which means:
Use a single-quoted string as the Awk script.
Pass any shell-variable values to the script via Awk's -v option to create Awk variables.
If we put it all together:
a=$(awk -F'[: ]' -v fruit="$fruit" '$2 == fruit' data.txt)
Note that I've used modern command-substitution syntax $(...), which is preferable to legacy syntax `...`.
When I use AWK print command outside shell it is working perfectly. Below is content of the file (sample.txt) which is comma separated.
IROG,1245,OUTO,OTUG,USUK
After, executing below command outside shell I get IROG as output.
cat sample.txt | awk -F, '{print $1}' > data.txt
Below is inside the shell script
my $HOME ='home/tmp/stephen';
my $DATA ="$HOME/data.txt";
my $SAMPLE ="$HOME/sample.txt";
`cat $SAMPLE | awk -F, '{print $1}' > $DATA`;
But here i get the same content as in original file instead of 1st column.
output is IROG,1245,OUTO,OTUG,USUK
but I expect only IROG. Can someone advise where I am wrong here?
The $1 inside your backticks expression is being expanded by perl before being executed by the shell. Presumably it has no value, so your awk command is simply {print }, which prints the whole record. You should escape the $ to prevent this from happening:
`awk -F, '{print \$1}' "$SAMPLE" > "$DATA"`;
Note that I have quoted your variables and also removed your useless use of cat.
If you mean to use a shell script, as opposed to a perl one (which is what you've currently got), you can do this:
home=home/tmp/stephen
data="$home/data.txt"
sample="$home/sample.txt"
awk -F, '{print $1}' "$sample" > "$data"
In the shell, there must be no spaces in variable assignments. Also, it is considered bad practice to use UPPERCASE variable names, as you risk overwriting the ones used internally by the shell. Furthermore, it is considered good practice to use double quotes around variable expansions to prevent problems related to word splitting and glob expansion.
There are a few ways that you could trim the leading whitespace from your first field. One would be to use sub to remove it:
awk -F, '{sub(/^ */, ""); print $1}'
This removes any space characters from the start of the line. Again, remember to escape the $ if doing this within backticks in perl.
Below is my sample data, in a file called 'tt':
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$"
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$"
I want to replace "HISTOGRAMBIN_#####002__UDBDI02$" with "HISTOGRAMBIN_#####002__UDBDI02$_MIG"
Why does this awk command intended to do a substitution not work?
awk '/HISTOGRAMBIN_#####002__UDBDI02\$/ {sub("'HISTOGRAMBIN_#####002__UDBDI02\$'","'HISTOGRAMBIN_#####002__UDBDI02\$_MIG'")} {print}' tt
EDIT: The awk command here is used to replace the object name picked from another file. $ can be anywhere in the name, but i want _mig to be at the end. My actual awk command which i use in my script is below:
awk '/"'${TAB_NAME}'"/{if (M==""){sub("'${TAB_NAME}'","'${TAB_NAME}_${TAB_EXT}'");M=1}}{print}' filename
I have to use "" to expand awk to use shell variables and I just want to replace the first occurrence.
in your awk line, the quotes were not correctly used. do
awk '{sub(/H....\$/, "replacement")}7' file
You could use sed for this purpose and you don't need to replace the whole string just replacing $ with $_MG is enough,
$ sed -r 's/^([^$]*\$)(.*)$/\1_MG\2/g' file
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
OR
A simpler one,
$ sed 's/\$/$_MG/g' file
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
awk '{sub(/\$/,"$_MG")}1' file
sed 's/\$/$_MG/' file
When you wrote:
awk '...sub("'HISTOGRAMBIN_#####002__UDBDI02\$'",...)...' file
The single quotes withing your sub() were causing your shell script to exit and re-enter awk the awk script specification so the HISTOGRAMBIN_#####002__UDBDI02\$ was actully being interpreted by shell first and THEN because you had the RE enclosed in double quotes instead of RE delimiters (/) the resulting string was being interpreted by awk twice, once when the script was read and then again when it was executed. So - you'd have had to escape that $ at least 3 times before awk stood a chance of it still being escaped when the script was executed.
Thanks Guys,
I got my awk command working
awk '/'${TAB_NAME}'/ {if (M==""){sub(/'$TAB_NAME'/,"'${TAB_NAME}_${TAB_EXT}'");M=1}}{print}' tt
Thanks kent, Ed Morton and avinash-raj for help.
I have a shell script that constructs an awk program as a string then pass that string to awk. This is because I want to use values of shell variables in the awk program.
My code looks like this:
awk_prog="'{if (\$4~/$shell_var/) print \$1,\$2}'"
echo $awk_prog
awk $awk_prog $FILENAME
However, when I pass the string to awk, I always get the error:
'{if ($4~/regex/) print $1,$2}'
awk: '{if
awk: ^ invalid char ''' in expression
What does that error message mean? I tried the -F: switch but it does not help. How can I settle this issue?
Thank you.
This is caused by shell quoting. The following will work:
awk_prog="{ if (\$4 ~ /$shell_var/) print \$1, \$2 }"
echo "$awk_prog"
awk "$awk_prog" $FILENAME
When you run awk '{ print }' foo from the command line, the shell interprets and removes the quotes around the program so awk receives two arguments - the first is the program text and the second is the filename foo. Your example was sending awk the program text '{if ...}' which is invalid syntax as far as awk is concerned. The outer quotes should not be present.
In the snippet that I gave above, the shell uses the quotes in the awk_prog= line to group the contents of the string into a single value and then assigns it to the variable awk_prog. When it executes the awk "$awk_prog"... line, you have to quote the expansion of $awk_prog so awk receives the program text as a single argument.
There's another way to get your shell variable into awk -- use awk's -v option:
awk -v pattern="$shell_var" '$4 ~ pattern {print $1, $2}' "$FILENAME"
Use -v multiple times if you have several variables to pass to awk.
If you truly want to hold your awk program in a shell variable, build it up using printf:
awk_script="$( printf '$4 ~ /%s/ {print $1, $2}' "$shell_var" )"
awk "$awk_script" "$FILENAME"
Note the use of quotes in the printf command: single quotes around the template to protect the dollar signs you want awk to interpret, double quotes for shell variables.
Another (IMO simpler) solution which (I think) addresses what you are intuitively trying to do is simply to use eval. You want the shell to behave as if you had literally typed:
awk '{if ($4~/foo/) print $1,$2}' path
(where foo and path are the literal contents of $shell_var and $FILENAME). To make that happen, just slap an eval on the front of your last line (and perhaps quotes for good measure, but they aren't necessary in this case) so that your last line is:
eval "awk $awk_prog $FILENAME"