I have a data in Unix which were fetched by a command and prints:
line01
line02
line03
line04
line05
line06
line07
line08
line09
line10
line11
line12
and I wanted to sort it out such that the lines 10 to 12 are above lines 1 to 9. like this:
line10
line11
line12
line01
line02
line03
line04
line05
line06
line07
line08
line09
i tried using
<command that fetches the data> | awk 'NR>=10 || NR<=9'
and
<command that fetches the data> | sed -n -e '4,5p' -e '1,3p'
but it still display in a sorted order. i'm new to unix so i don't know how to properly use awk/sed.
PS. These data are stored in a variable which will then be processed by another command. so i needed it to be sorted that way so that line 10-12 will be processed first. :)
Use head and tail:
$ tail -n 2 file && head -n 3 file
name4
name5
name1
name2
name3
Your awk and sed approach do not work because you are just saying: print lines number X, Y and Z, and they will do so as soon as they find any of them. If you wanted to use these tools, you would need to read the file first, storing its content, and then print it.
$ awk -v OFS="\n" '{a[NR]=$0} END {print a[4], a[5], a[1], a[2], a[3]}' file
name4
name5
name1
name2
name3
Or even give the order as a variable:
awk -v order="4 5 1 2 3"
'BEGIN {split(order,lines)}
{a[NR]=$0}
END {for (i=1;i<=length(lines);i++) print a[lines[i]]}' file
If you want to give the order of the lines as an argument, you can use process substitution saying awk '...' <(command) file and working with FNR/NR to distinguish between the input and the file
Or you can use - to read from stdin as first file:
echo "4 5 1 2 3" | awk 'FNR==NR {n=split($0,lines); next}
{a[FNR]=$0}
END {for (i=1;i<=n;i++) print a[lines[i]]}' - file
As one-liner:
$ echo "4 5 1 2 3" | awk 'FNR==NR {n=split($0,lines); next} {a[FNR]=$0} END {for (i=1;i<=n;i++) print a[lines[i]]}' - a
This might work for you (GNU sed):
sed '1h;2,9H;1,9d;12G' file
Replace the hold space with line 1, then append lines 2 to 9 to the hold space and delete lines 1 thru 9. Print all other lines normally but on line 12 append the lines stored in the hold space to the pattern space.
Using sort actually:
$ sort -r -s -k 1.5,1.5 /tmp/lines
line10
line11
line12
line01
line02
line03
line04
line05
line06
line07
line08
line09
The -k 1.5,1.5 means I'm using only 5th character of first word for sorting. -r means reverse order and -s means stable - leaving lines that have same 5th character in the same order.
Related
In my shell script, one of the variable contains set of lines. I have a requirement to get the two lines info at single iteration in which my awk needs it.
var contains:
12 abc
32 cdg
9 dfk
98 nhf
43 uyr
5 ytp
Here, In a loop I need line1, line2[i.e 12 abc \n 32 cdg] content and next iteration needs line2, line3 [32 cdg \n 9 dfk] and so on..
I tried to achieve by
while IFS= read -r line
do
count=`echo ${line} | awk -F" " '{print $1}'`
id=`echo ${line} | awk -F" " '{print $2}'`
read line
next_id=`echo ${line} | awk -F" " '{print $2}'`
echo ${count}
echo ${id}
echo ${next_id}
## Here, I have some other logic of awk..
done <<< "$var"
It's reading line1, line2 at first iteration. At second iteration it's reading line3, line4. But, I required to read line2, line3 at second iteration. Can anyone please sort out my requirement.
Thanks in advance..
Don't mix a shell script spawing 3 separate subshells for awk per-iteration when a single call to awk will do. It will be orders of magnitude faster for large input files.
You can group the messages as desired, just by saving the first line in a variable, skipping to the next record and then printing the line in the variable and the current record through the end of the file. For example, with your lines in the file named lines, you could do:
awk 'FNR==1 {line=$0; next} {print line"\n"$0"\n"; line=$0}' lines
Example Use/Output
$ awk 'FNR==1 {line=$0; next} {print line"\n"$0"\n"; line=$0}' lines
12 abc
32 cdg
32 cdg
9 dfk
9 dfk
98 nhf
98 nhf
43 uyr
43 uyr
5 ytp
(the additional line-break was simply included to show separation, the output format can be changed as desired)
You can add a counter if desired and output the count via the END rule.
The solution depends on what you want to do with the two lines.
My first thought was something like
sed '2,$ s/.*/&\n&/' <<< "${yourvar}"
But this won't help much when you must process two lines (I think | xargs -L2 won't help).
When you want them in a loop, try
while IFS= read -r line; do
if [ -n "${lastline}" ]; then
echo "Processing lines starting with ${lastline:0:2} and ${line:0:2}"
fi
lastline="${line}"
done <<< "${yourvar}"
What's the easiest/quickest way to interleave the lines of two (or more) text files? Example:
File 1:
line1.1
line1.2
line1.3
File 2:
line2.1
line2.2
line2.3
Interleaved:
line1.1
line2.1
line1.2
line2.2
line1.3
line2.3
Sure it's easy to write a little Perl script that opens them both and does the task. But I was wondering if it's possible to get away with fewer code, maybe a one-liner using Unix tools?
paste -d '\n' file1 file2
Here's a solution using awk:
awk '{print; if(getline < "file2") print}' file1
produces this output:
line 1 from file1
line 1 from file2
line 2 from file1
line 2 from file2
...etc
Using awk can be useful if you want to add some extra formatting to the output, for example if you want to label each line based on which file it comes from:
awk '{print "1: "$0; if(getline < "file2") print "2: "$0}' file1
produces this output:
1: line 1 from file1
2: line 1 from file2
1: line 2 from file1
2: line 2 from file2
...etc
Note: this code assumes that file1 is of greater than or equal length to file2.
If file1 contains more lines than file2 and you want to output blank lines for file2 after it finishes, add an else clause to the getline test:
awk '{print; if(getline < "file2") print; else print ""}' file1
or
awk '{print "1: "$0; if(getline < "file2") print "2: "$0; else print"2: "}' file1
#Sujoy's answer points in a useful direction. You can add line numbers, sort, and strip the line numbers:
(cat -n file1 ; cat -n file2 ) | sort -n | cut -f2-
Note (of interest to me) this needs a little more work to get the ordering right if instead of static files you use the output of commands that may run slower or faster than one another. In that case you need to add/sort/remove another tag in addition to the line numbers:
(cat -n <(command1...) | sed 's/^/1\t/' ; cat -n <(command2...) | sed 's/^/2\t/' ; cat -n <(command3) | sed 's/^/3\t/' ) \
| sort -n | cut -f2- | sort -n | cut -f2-
With GNU sed:
sed 'R file2' file1
Output:
line1.1
line2.1
line1.2
line2.2
line1.3
line2.3
Here's a GUI way to do it: Paste them into two columns in a spreadsheet, copy all cells out, then use regular expressions to replace tabs with newlines.
cat file1 file2 |sort -t. -k 2.1
Here its specified that the separater is "." and that we are sorting on the first character of the second field.
$cat file
1
2
3
4
5
6
7
8
9
0
I want to print the value of last row.
$awk '{print $NR}' file
1
Why the output is not 0?
Unlike sed, awk does not have a way to specify the last line. A work-around is:
$ awk '{line=$0} END{print line}' file
0
Discussion
Let's look at your command at see what it actually does. Consider this test file:
$ cat testfile
a b c
A B C
i ii iii
Now, let's run your command:
$ awk '{print $NR}' testfile
a
B
iii
As you can see, print $NR prints the diagonal. In other words, on line number NR, it prints field number NR. So, on the first line, NR=1, the command print $NR prints the first field. On the second line, NR=2, the command print $NR prints the second field. And so on.
Use following code, which will print the last line of any Input_file. Here END section is the out of the box awk keyword which is used to execute the commands/statements after main section. So I am simply printing the line in END section which will print the last line.
awk 'END{print $0}' Input_file
OR
awk 'END{print}' Input_file
I have this file content:
2450TO3450
3800
4500TO4560
And I would like to obtain something of this sort:
2450
2454
2458
...
3450
3800
4500
4504
4508
..
4560
Basically I would need a one liner in sed/awk that would read the values on both sides of the TO separator and inject those in a seq command or do the loop on its own and dump it in the same file as a value per line with an arbitrary increment, let's say 4 in the example above.
I know I can use several one temp file, go the read command and sorts, but I would like to do it in a one liner starting with cat filename | etc. as it is already part of a bigger script.
Correctness of the input is guaranteed so always left side of TOis smaller than bigger side of it.
Thanks
Like this:
awk -F'TO' -v inc=4 'NF==1{print $1;next}{for(i=$1;i<=$2;i+=inc)print i}' file
or, if you like starting with cat:
cat file | awk -F'TO' -v inc=4 'NF==1{print $1;next}{for(i=$1;i<=$2;i+=inc)print i}'
Something like this might work:
awk -F TO '{system("seq " $1 " 4 " ($2 ? $2 : $1))}'
This would tell awk to system (execute) the command seq 10 4 10 for lines just containing 10 (which outputs 10), and something like seq 10 4 40 for lines like 10TO40. The output seems to match your example.
Given:
txt="2450TO3450
3800
4500TO4560"
You can do:
echo "$txt" | awk -F TO '{$2<$1 ? t=$1 : t=$2; for(i=$1; i<=t; i++) print i}'
If you want an increment greater than 1:
echo "$txt" | awk -F TO -v p=4 '{$2<$1 ? t=$1 : t=$2; for(i=$1; i<=t; i+=p) print i}'
Give a try to this:
sed 's/TO/ /' file.txt | while read first second; do if [ ! -z "$second" ] ; then seq $first 4 $second; else printf "%s\n" $first; fi; done
sed is used to replace TO with space char.
read is used to read the line, if there are 2 numbers, seq is used to generate the sequence. Otherwise, the uniq number is printed.
This might work for you (GNU sed):
sed -r 's/(.*)TO(.*)/seq \1 4 \2/e' file
This evaluates the RHS of the substitution command if the LHS contains TO.
I have a text file like this:
1.2.3.t
1.2.4.t
complete
I need to print the last non blank line and two line to last as two variable. the output should be:
a=1.2.4.t
b=complete
I tried this for last line:
b=awk '/./{line=$0} END{print line}' myfile
but I have no idea for a.
grep . file | tail -n 2 | sed 's/^ *//;1s/^/a=/;2s/^/b=/'
Output:
a=1.2.4.t
b=complete
awk to the rescue!
$ awk 'NF{a=b;b=$0} END{print "a="a;print "b="b}' file
a=1.2.4.t
b=complete
Or, if you want to the real variable assignment
$ awk 'NF{a=b;b=$0} END{print a, b}' file
| read a b; echo "a="$a; echo "b="$b
a=1.2.4.t
b=complete
you may need -r option for read if you have backslashes in the values.