I'm trying to learn to use awk but it's not behaving how I expect. Here's my trouble:
$ echo "Hello brave new world" | awk "{print $1}"
Hello brave new world
I expected to see "Hello", as this is the first field. Why don't the spaces count as field delimiters?
It's because Bash is interpreting $1 as referring to the first shell argument, so it replaces it with its value. Since, in your case, that parameter is unset, $1 just gets replaced with the empty string; so your AWK program is actually just {print }, which prints the whole line.
To prevent Bash from doing this, wrap your AWK program in single-quotes instead of double-quotes:
echo "Hello brave new world" | awk '{print $1}'
or
echo 'Hello brave new world' | awk '{print $1}'
echo "Hello brave new world" | awk '{print $1}'
Use single quotes around the awk program, otherwise $1 gets translated as the shell variable $1
To print the entire line, run this shell-command:
echo "Hi, this is my first answer on stackoverflow" | awk '{ print $0 }'
output:
Hi, this is my first answer on stackoverflow
You can use $0 for full line.
$ cat <<EOF | head -1 | sed "s/\t/\n/g" | awk 'BEGIN { cont=0} { print ++cont " " $0}'
> First Name His/Her Phone Electronic mail address Acquaintance or Friend
Amelia 555-5553 amelia.zodiacusque#gmail.com F
Anthony 555-3412 anthony.asserturo#hotmail.com A
Becky 555-7685 becky.algebrarum#gmail.com A
EOF
1 First Name
2 His/Her Phone
3 Electronic mail address
4 Acquaintance or Friend
Data comes from https://www.gnu.org/software/gawk/manual/gawk.html#Sample-Data-Files
but here is tab separated.
See also https://www.gnu.org/software/gawk/manual/gawk.html#Very-Simple
Related
Hi I have a shell command like this.
s3=$(awk 'BEGIN{ print "S3 bucket path" }
/Executing command\(queryId/{ sub(/.*queryId=[^[:space:]]+: /,""); q=$0 }
/s3:\/\//{ print "," $10 }' OFS=',' hive-server2.log)
The output of the above command like this.
echo $s3
2018-02-21T17:58:22,
2018-02-21T17:58:26,
2018-02-21T18:05:33,
2018-02-21T18:05:34
I want to select the last line only. I need the last output like this.
2018-02-21T18:05:34
I tried like this.
awk -v $s3 '{print $(NF)}'
Not working.Any help will be appreciated.
In general, command | tail -n 1 prints the last line of the output from command. However, where command is of the form awk '... { ... print something }' you can refactor to awk '... { ... result = something } END { print result }' to avoid spawning a separate process just to discard the other output. (Conversely, you can replace awk '/condition/ { print something }' | head -n 1 with awk '/condition/ { print something; exit }'.)
If you already have the result in a shell variable s3 and want to print just the last line, a parameter expansion echo "${s3##*$'\n'}" does that. The C-style string $'\n' to represent a newline is a Bash extension, and the parameter expansion operator ## to remove the longest matching prefix isn't entirely portable either, so you should make sure the shebang line says #!/bin/bash, not #!/bin/sh
Notice also that $s3 without quotes is an error unless you specifically require the shell to perform whitespace tokenization and wildcard expansion on the value. You should basically always use double quotes around variables except in a couple of very specific scenarios.
Your Awk command would not work for two reasons; firstly, as explained in the previous paragraph, you are setting s3 to the first token of the variable, and the second is your Awk script (probably a syntax error). In more detail, you are basically running
awk -v s3=firstvalue secondvalue thirdvalue '{ print $(NF) }'
^ value ^ script to run ^ names of files ...
where you probably wanted to say
awk -v s3=$'firstvalue\nsecondvalue\nthirdvalue' '{ print $(NF) }'
But even with quoting, your script would set v to something but then tell Awk to (ignore the variable and) process standard input, which on the command line leaves it reading from your terminal. A fixed script might look like
awk 'END { print }' <<<"$s3"
which passes the variable as standard input to Awk, which prints the last line. The <<<value "here string" syntax is also a Bash extension, and not portable to POSIX sh.
much simple way is
command | grep "your filter" | tail -n 1
or directly
command | tail -n 1
You could try this:
echo -e "This is the first line \nThis is the second line" | awk 'END{print}'
another approach can be, processing the file from the end and exiting after first match.
tac file | awk '/match/{print; exit}'
Hi you can do it just by adding echo $s3 | sed '$!d'
s3=$(awk 'BEGIN{ print "S3 bucket path" }/Executing command\(queryId/{ sub(/.*queryId=[^[:space:]]+: /,""); q=$0 } /s3:\/\//{ print "," $10 }' OFS=',' hive-server2.log)
echo $s3 | sed '$!d'
It will simply print:-
2018-02-21T18:05:34
Hope this will help you.
I'm a newbie with very small and specific needs. I'm using awk to parse something and I need to generate uninterrupted lines of text assembled from several pieces in the original text. But awk inserts a newline in the output whenever I use a semicolon.
Simplest example of what I mean:
Original text:
1 2
awk command:
{ print $1; print $2 }
The output will be:
1
2
The thing is that I need the output to be a single line, and I also need to use the semicolons, because I have to do multiple actions on the original text, not all of them print.
Also, using ORS=" " causes a whole lot of different problems, so it's not an option.
Is there any other way that I can have multiple actions in the same line without newline insertion?
Thanks!
The newlines in the output are nothing to do with you using semicolons to separate statements in your script, they are because print outputs the arguments you give it followed by the contents of ORS and the default value of ORS is newline.
You may want some version of either of these:
$ echo '1 2' | awk '{printf "%s ", $1; printf "%s ", $2; print ""}'
1 2
$
$ echo '1 2' | awk -v ORS=' ' '{print $1; print $2; print "\n"}'
1 2
$
$ echo '1 2' | awk -v ORS= '{print $1; print " "; print $2; print "\n"}'
1 2
$
but it's hard to say without knowing more about what you're trying to do.
At least scan through the book Effective Awk Programming, 4th Edition, by Arnold Robbins to get some understanding of the basics before trying to program in awk or you're going to waste a lot of your time and learn a lot of bad habits first.
You have better control of the output if you use printf, e.g.
awk '{ printf "%s %s\n",$1,$2 }'
awk '{print $1 $2}'
Is the solution in this case
TL;DR
You're getting newlines because print sends OFS to standard output after each print statement. You can format the output in a variety of other ways, but the key is generally to invoke only a single print or printf statement regardless of how many fields or values you want to print.
Use Commas
One way to do this is to use a single call to print using commas to separate arguments. This will insert OFS between the printed arguments. For example:
$ echo '1 2' | awk '{print $1, $2}'
1 2
Don't Separate Arguments
If you don't want any separation in your output, just pass all the arguments to a single print statement. For example:
$ echo '1 2' | awk '{print $1 $2}'
12
Formatted Strings
If you want more control than that, use formatted strings using printf. For example:
$ echo '1 2' | awk '{printf "%s...%s\n", $1, $2}'
1...2
$ echo "1 2" | awk '{print $1 " " $2}'
1 2
I have a file a.text:
hello world
my world
hello universe
I want to print the complete string if the second word is "world":
[root#sc-rdops-vm18-dhcp-57-128:/var/log] cat a | awk -F " " '{if($2=="world") print $1}'
hello
my
But the output which I want is:
[root#sc-rdops-vm18-dhcp-57-128:/var/log] cat a | awk -F " " '{if($2=="world") print <Something here>}'
hello world
my world
Any pointers on how I can do this?
Thanks in advance.
awk '{if ($2=="world") {print}}' file
Output:
hello world
my world
First off, since you are writing a single if statement, you can use the awk 'filter{commands;}' pattern, like so
awk -F " " '$2=="world" { print <Something here> }'
To print the entire line you can use print $0
awk -F " " '$2=="world"{print $0}' file
which can be written as
awk -F " " '$2=="world"{print}' file
But {print} is the default action, so it can be omitted after the filter like this:
awk -F " " '$2=="world"' file
Or even without the -F option, since the space is the default FS value
awk '$2=="world"' file
If you want / have to use awk to solve your problem:
awk '$0~/world/' file.txt
If a line (i.e., $0) matches the string "world" (i.e., ~/world/) the entire line is printed
If you only want to check the second column for world:
awk '$2 == "world"' file.txt
How do print my full name with the following?
awk -F: '($1==U){print $5}' U=$LOGNAME /etc/passwd
Per example, but with the echo command with some words on sides:
For example,
Hello Diogo Saraiva, happy to see you again
Where Diogo Saraiva is my full name which I have in Ubuntu records.
I tried some things, but I have not accomplished that script...
Another thing: Why, when I issue awk -F: '($1==U){print $5}' U=$LOGNAME /etc/passwd, is Diogo Saraiva,,, shown instead of Diogo Saraiva?
This happens too with:
grep $USER /etc/passwd | awk 'BEGIN { FS=":" } { print $5 }'
I need for my script to declare a variable, "like", with the command, and then echo that variable "like" various times along my script.
To output the string you want you can use this command.
awk -F: -v U="$LOGNAME" '$1==U{print "Hello " $5 ", happy to see you again"}' /etc/passwd
If awk -F: -v U="$LOGNAME" '$1==U{print $5}' /etc/passwd is outputting Diogo Saraiva,,, then that is what is in your /etc/passwd file for that field.
To save the output from a command in a variable just use:
var=$(command)
And remember to quote the variable when you use it:
echo "Hello $var, happy to see you again"
I am trying to make a single line ssh call from a ruby script. My script takes a hostname, and then sets out to return the hostname's machine info.
return_value = %x{ ssh #{hostname} "#{number_of_users}; #{number_of_processes};
#{number_of_processes_running}; #{number_of_processes_sleeping}; "}
Where the variables are formatted like this.
number_of_users = %Q(users | wc -w | cat | awk '{print "Number of Users: "\$1}')
number_of_processes = %Q(ps -el | awk '{print $2}' | wc -l | awk '{print "Number of Processes: "$1}')
I have tried both %q, %Q, and just plain "" and I cannot get the awk to print anything before the output. I either get this error (if I include the colon)
awk: line 1: syntax error at or near :
or if I don't include the slash in front of $1 I just get empty output for that line. Is there any solution for this? I thought it might be because I was using %q, but it even happens with just double quotes.
Use backticks to capture the output of the command and return the output as a string:
number_of_users = `users | wc -w | cat | awk '{print "Number of Users:", $1}'`
puts number_of_users
Results on my system:
48
But you can improve your pipeline:
users | awk '{ print "Number of Users:", NF }'
ps -e | awk 'END { print "Number of Processes:", NR }'
So the solution to this problem is:
%q(users | wc -w | awk '{print \"Number of Users: \"\$1}')
Where you have to use %q, not %, not %Q, and not ""
You must backslash double quotes and the dollar sign in front of any awk variables
If somebody could improve upon this answer by explaining why, that would be most appreciated
Though as Steve pointed out I could have improved my code using users | awk '{ print \"Number of Users:\", NF }'
In which case there is no need to backslash the NF.