I am having a little trouble with using a variable and printing the 2nd field with the awk command. I am attempting to grab a number from a value in a file. The value in the file looks like
MAX=10000 (I want the Number only), I am passing this into a variable in a script so in the script I have variables
parm_file=ParmFiles/Parmfile.parm
session=s_session_value
OLD_MAX_SEQ_NR=`awk -F '=' "/$session/ {getline; print $2}" < $parm_file`
because I have double quotes to identify the $session variable, it is taking the $2 as a variable too, and so it is just printing the whole line, instead of the second field.
I've tried also to pass the variable into the awk command like
OLD_MAX_SEQ_NR=`awk -F '=' \
-v var="$session" \
'/var/ {getline; print $2}' < $parm_file`
But it does not seem to be putting the variable where var is. I have even tried hard coding the -v var="s_session_value" and it does not work.
I can't figure out a way to make the command look at the $2 as it normally does instead of a variable. Any help would be greatly appreciated.
Try this:
parm_file=ParmFiles/Parmfile.parm
session=s_session_value
OLD_MAX_SEQ_NR=$(
awk -F'=' -v pat="$session" \
'$0 ~ pat {getline; print $2}' < "$parm_file"
)
You need to pass shell variables to awk by defining an awk variable using -v.
Using variable inside /../ is taken as literal. So use $0~var_name construct.
Using back-ticks is deprecated. Use command substitution $(..)
Quote your variables.
It's a bit tricky without a sample line of the parm file. But I don't understand why you don't use cut, it makes it much easier?
OLD_MAX_SEQ_NR=$(grep "$session" "$parm_file" | cut -d= -f2)
Related
Does awk support using a different delimiter character than ' in print? e.g. Instead of awk '{print $1}', something like awk -d # #{print $1}#
I was actually looking at the C source code and it's a pretty short program; is there an alternative version that allow that?
It can't: The ' isn't passed to awk; instead, it's understood by the shell itself. Thus, when you run awk '{print $1}', what you're actually calling at the OS level is something like:
/* this is C syntax, so the double-quotes are C quotes; only their contents are literal */
execvp("awk", { "awk", "{print $1}", NUL });
Notably, the single-quotes aren't there at all any more -- they were parsed out by the shell when it understood them as instructions for how it should break the command into an argument list.
To see it in another way, consider if you put your script in a separate file and called
awk -f my.awk
The contents of my.awk would simply be
{print $1}
not
'{print $1}'
The single quotes are only used by the shell to ensure that the script is passed literally to awk, rather than being subject to any particular shell processing that could change the script before awk could read it.
I am new to AWK and trying to write some code where I can delete all files in a directory apart from the newest N number.
My code works if I use a hard coded number instead of a variable.
Works:
delete=`ls -t | awk 'NR>3'`
echo $delete
Does Not Work:
amount=3
delete=`ls -t | awk 'NR>$amount'`
echo $delete
I know the problem lies somewhere with the bash variable not being recognised as an awk variable however I do not know how to correct.
I have tried variations of code to fix this such as below, however I am still at a loss.
amount=3
delete=`ls -t | awk -v VAR=${amount} 'NR>VAR'`
echo $delete
Could you please advise what the correct code is ?
Shells don't expand anything inside single quotes.
Either:
amount=3
delete=$(ls -t | awk "NR>$amount")
or:
amount=3
delete=$(ls -t | awk -v amount=$amount 'NR > amount')
Be aware that parsing the output of ls is fraught if your file names are not limited to the portable file name character set. Spaces, newlines, and other special characters in the file name can wreck the parsing.
The simplest fix is to use double quotes instead of single. Single quotes prevent the shell from interpolating the variable $amount in the quoted string.
amount=3
ls -t | awk "NR>$amount"
I would not use a variable to capture the result. If you do use one, you need to quote it properly when interpolating it.
amount=3
delete=$(ls -t | awk -v VAR="$amount" 'NR>VAR')
echo "$delete"
Note that this is basically identical to your second attempt, which should have worked, modulo the string quoting issues.
When I use AWK print command outside shell it is working perfectly. Below is content of the file (sample.txt) which is comma separated.
IROG,1245,OUTO,OTUG,USUK
After, executing below command outside shell I get IROG as output.
cat sample.txt | awk -F, '{print $1}' > data.txt
Below is inside the shell script
my $HOME ='home/tmp/stephen';
my $DATA ="$HOME/data.txt";
my $SAMPLE ="$HOME/sample.txt";
`cat $SAMPLE | awk -F, '{print $1}' > $DATA`;
But here i get the same content as in original file instead of 1st column.
output is IROG,1245,OUTO,OTUG,USUK
but I expect only IROG. Can someone advise where I am wrong here?
The $1 inside your backticks expression is being expanded by perl before being executed by the shell. Presumably it has no value, so your awk command is simply {print }, which prints the whole record. You should escape the $ to prevent this from happening:
`awk -F, '{print \$1}' "$SAMPLE" > "$DATA"`;
Note that I have quoted your variables and also removed your useless use of cat.
If you mean to use a shell script, as opposed to a perl one (which is what you've currently got), you can do this:
home=home/tmp/stephen
data="$home/data.txt"
sample="$home/sample.txt"
awk -F, '{print $1}' "$sample" > "$data"
In the shell, there must be no spaces in variable assignments. Also, it is considered bad practice to use UPPERCASE variable names, as you risk overwriting the ones used internally by the shell. Furthermore, it is considered good practice to use double quotes around variable expansions to prevent problems related to word splitting and glob expansion.
There are a few ways that you could trim the leading whitespace from your first field. One would be to use sub to remove it:
awk -F, '{sub(/^ */, ""); print $1}'
This removes any space characters from the start of the line. Again, remember to escape the $ if doing this within backticks in perl.
How to use awk variable in search?
Name="jony"
awk -v name="$Name" '/name/ {print $0}' file
this will search for string name, not for $Name which is actually jony.
Correct, awk won't recogize variables in / /. You can do:
Name="jony"
awk -v name="$Name" '$0 ~ name' file
Since print is awk's default behavior we can avoid using it here.
Hope I understood problem correctly:
Why wont you try following one:
awk '/'"$Name"'/ { print } ' testfile
When writing an AWK one-liner, you could quote the script with either the single quotes or double quotes. In the latter case the shell does all the substitution directly so that you do not need to pass the variable into the script via -v option:
Name="jony"
awk "/$Name/" file
# this works. after shell has performed substitutions, the line looks like
awk "/jony/" file
[bad!] Or even without quotes if the name does not contain spaces:
awk /$Name/ file
All the simplicity vanishes as soon as you want to use $ in the script, including awk special variables that use $0, $1, etc, because you will have to escape the dollar sign to prevent shell variable expansion.
awk "/$Name/ {print \$0}"
In addition you will have to escape the double quotes to add literal text to the script. Looks clumsy:
awk "/$Name/ {print \"Found in: \" \$0}"
To crown it all, negating regular expression with double quotes will cause a shell error:
awk "!/$Name/"
#error> ... event not found ...
The error will happen if $Name itself contains ! sign. This makes using double quotes unreliable.
So, to be on the safe side, prefer single quotes :)
I have a shell script that constructs an awk program as a string then pass that string to awk. This is because I want to use values of shell variables in the awk program.
My code looks like this:
awk_prog="'{if (\$4~/$shell_var/) print \$1,\$2}'"
echo $awk_prog
awk $awk_prog $FILENAME
However, when I pass the string to awk, I always get the error:
'{if ($4~/regex/) print $1,$2}'
awk: '{if
awk: ^ invalid char ''' in expression
What does that error message mean? I tried the -F: switch but it does not help. How can I settle this issue?
Thank you.
This is caused by shell quoting. The following will work:
awk_prog="{ if (\$4 ~ /$shell_var/) print \$1, \$2 }"
echo "$awk_prog"
awk "$awk_prog" $FILENAME
When you run awk '{ print }' foo from the command line, the shell interprets and removes the quotes around the program so awk receives two arguments - the first is the program text and the second is the filename foo. Your example was sending awk the program text '{if ...}' which is invalid syntax as far as awk is concerned. The outer quotes should not be present.
In the snippet that I gave above, the shell uses the quotes in the awk_prog= line to group the contents of the string into a single value and then assigns it to the variable awk_prog. When it executes the awk "$awk_prog"... line, you have to quote the expansion of $awk_prog so awk receives the program text as a single argument.
There's another way to get your shell variable into awk -- use awk's -v option:
awk -v pattern="$shell_var" '$4 ~ pattern {print $1, $2}' "$FILENAME"
Use -v multiple times if you have several variables to pass to awk.
If you truly want to hold your awk program in a shell variable, build it up using printf:
awk_script="$( printf '$4 ~ /%s/ {print $1, $2}' "$shell_var" )"
awk "$awk_script" "$FILENAME"
Note the use of quotes in the printf command: single quotes around the template to protect the dollar signs you want awk to interpret, double quotes for shell variables.
Another (IMO simpler) solution which (I think) addresses what you are intuitively trying to do is simply to use eval. You want the shell to behave as if you had literally typed:
awk '{if ($4~/foo/) print $1,$2}' path
(where foo and path are the literal contents of $shell_var and $FILENAME). To make that happen, just slap an eval on the front of your last line (and perhaps quotes for good measure, but they aren't necessary in this case) so that your last line is:
eval "awk $awk_prog $FILENAME"