Reverse input order with sed - bash

I have a file, lets call it 'a.txt' and this file contains the following text line
do to what
I'm wondering what the SED command is to reverse the order of this text to make it look like
what to do
Do I have to do some sort of append? Like append 'do' to 'to' so it would look like
to ++ do (used ++ just to make it clear)

I know tac can do something related
$ cat file
do to what
$ tac -s' ' file
what to do $
Where the -s defines the separator, which is by default a newline.

I would use awk to do this:
awk '{ for (i=NF; i>=1; i--) printf (i!=1) ? $i OFS : $i "\n" }' file.txt
Results:
what to do
EDIT:
If you require a one-liner to modify your file "in-place", try:
{ rm file.txt && awk '{ for (i=NF; i>=1; i--) printf (i!=1) ? $i OFS : $i "\n" }' > file.txt; } < file.txt

sed answer
As this question was tagged sed, my 1st answer was:
First (using arbitraty _ to mark viewed spaces, when a.txt contain do to what:
sed -e '
:a;
s/\([^_]*\) \([^ ]*\)/\2_\1/;
ta;
y/_/ /;
' a.txt
what to do
than, when a.txt contain do to to what:
sed -e '
:a;
s/^\(\|.* \)\([^+ ]\+\) \2\([+]*\)\(\| .*\)$/\1\2\3+\4/g;
ta;
:b;
s/\([^_]*\) \([^ ]*\)/\2_\1/;
tb;
y/_/ /;
' <<<'do to to to what'
what to++ do
There is one + for each supressed duplicated word:
sed -e ':a;s/^\(\|.* \)\([^+ ]\+\) \2\([+]*\)\(\| .*\)$/\1\2\3+\4/g;ta;
:b;s/\([^_]*\) \([^ ]*\)/\2_\1/;tb;
y/_/ /;' <<<'do do to what what what what'
what+++ to do+
bash answer
But as there is a lot of people searching for simple bash solutions, there is a simple way:
xargs < <(uniq <(tac <(tr \ \\n <<<'do do to what what what what')))
what to do
this could be written:
tr \ \\n <<<'do do to what what what what' | tac | uniq | xargs
what to do
or even with some bash scripting:
revcnt () {
local wrd cnt plut out="";
while read cnt wrd; do
printf -v plus %$((cnt-1))s;
out+=$wrd${plus// /+}\ ;
done < <(uniq -c <(tac <(tr \ \\n )));
echo $out
}
Will do:
revcnt <<<'do do to what what what what'
what+++ to do+
Or as pure bash
revcnt() {
local out i;
for ((i=$#; i>0; i--))
do
[[ $out =~ ${!i}[+]*$ ]] && out+=+ || out+=\ ${!i};
done;
echo $out
}
where submited string have to be submitted as argument:
revcnt do do to what what what what
what+++ to do+
Or if prossessing standard input (or from file) is required:
revcnt() {
local out i arr;
while read -a arr; do
out=""
for ((i=${#arr[#]}; i--; 1))
do
[[ $out =~ ${arr[i]}[+]*$ ]] && out+=+ || out+=\ ${arr[i]};
done;
echo $out;
done
}
So you can process multiple lines:
revcnt <<eof
do to what
do to to to what
do do to what what what what
eof
what to do
what to++ do
what+++ to do+

This might work for you (GNU sed):
sed -r 'G;:a;s/^\n//;t;s/^(\S+|\s+)(.*)\n/\2\n\1/;ta' file
Explanation:
G add a newline to the end of the pattern space (PS)
:a loop name space
s/^\n//;t when the newline is at the front of the PS, remove it and print line
s/^(\S+|\s+)(.*)\n/\2\n\1/;ta insert either a non-space or a space string directly after the newline and loop to :a
The -r switch makes the regexp easier-on-the-eye (grouping (...), alternation ...|... and the metacharacter for one-or-more + are relieved of the need of a backslash prefix).
Alternative:
sed -E 'G;:a;s/^(\S+)(\s*)(.*\n)/\3\2\1/;ta;s/.//' file
N.B. To reverse the line, adapt the above solution to:
sed -E 'G;:a;/^(.)(.*\n)/\2\1/;ta;s/.//' file

May be you would like perl for this:
perl -F -lane '#rev=reverse(#F);print "#rev"' your_file

As Bernhard said, tac can be used here:
#!/usr/bin/env bash
set -eu
echo '1 2 3
2 3 4
3 4 5' | while IFS= read -r; do
echo -n "$REPLY " | tac -s' '
echo
done
$ ./1.sh
3 2 1
4 3 2
5 4 3
I believe my example is more helpful.

Related

How to turn a string into a modified hex representation?

i want to turn a string like
AaAa
into
a string like this
%<41>%<61>%<41>%<61>
Simple enough with the programming languages i am familar with, but with bash i can't get get the piping right to do what i am trying to do:
split string into char array
turn each char into hex
wrap each hex value into %<FF>
concat string
this is my current way which gets me half way there:
echo -n "AaAa" | od -A n -t x1
If you are already using od,
printf "%%<%s>" $(od -A n -t x1<<<"AaAa")
For an all-bash without od,
while read -r -N 1 c; do printf "%%<%02X>" "$( printf "%d" \'$c )"; done <<< AaAa
The downside of this approach is that it spawns a subshell for every character, and assumes ASCII/UTF8.
edit
#Shawn pointed out that you don't need the subshell -
while read -r -N 1 c; do printf "%%<%02X>" \'$c; done <<< AaAa
I noticed that these are leaving the string terminator in your output, though, and realized I could eliminate that and the read by assigning the data to a variable and using the built-in parsing tools.
$: x=AaAa && for((i=0;i<${#x};i++)); do printf "%%<%02X>" \'${x:i:1}; done; echo
%<41>%<61>%<41>%<61>
A simple Perl substitution would do the trick:
echo -n AaAa | perl -pe's/(.)/ sprintf "%%<%02X>", ord($1) /seg'
Shorter:
echo -n AaAa | perl -ne'printf "%%<%02X>", $_ for unpack "C*"'
In both cases, the output is the expected
%<41>%<61>%<41>%<61>
(No trailing line feed added. If you want one, append ; END { print "\n" }.)
You can pipe to sed to wrap each byte in %<> and then remove the whitespace.
echo -n "AaAa" | od -A n -t x1 | sed -E -e 's/[a-z0-9]+/%<&>/g' -e 's/ //g'
You could use perl:
echo -n AaAa | perl -ne 'for $c (split//) { printf("%%<%02X>", ord($c)); }'
Output
%<41>%<61>%<41>%<61>
Maybe awk
echo -n "AaAa" |
od -A n -t x1 |
awk 'BEGIN { ORS = "" } { for (i = 1; i <= NF; i+=1) print "%<"$i">"}'

Convert substring through command

Basically, how do I make a string substitution in which the substituted string is transformed by an external command?
For example, given the line 5aaecdab287c90c50da70455de03fd1e ./2015/01/26/GOPR0083.MP4, how to pipe the second part of the line (./2015/01/26/GOPR0083.MP4) to command xargs stat -c %.6Y and then replace it with the result so that we end up with 5aaecdab287c90c50da70455de03fd1e 1422296624.010000?
This can be done with a script, however a one-liner would be nice.
#!/bin/bash
hashtime()
{
while read longhex fname; do
echo "$longhex $(stat -c %.6Y "$fname")"
done
}
if [ $# -ne 1 ]; then
echo Usage: ${0##*/} infile 1>&2
exit 1
fi
hashtime < $1
exit 0
# one liner
awk 'BEGIN { args="stat -c %.6Y " } { printf "%s ", $1; cmd=args $2; system(cmd); }' infile
A one-liner using GNU sed, which will process the whole file:
sed -E "s/([[:xdigit:]]+) +(.*)/stat -c '\1 %.6Y' '\2'/e" file
or, using plain bash
while read -r hash pathname; do stat -c "$hash %.6Y" "$pathname"; done < file
It's typical to use awk sed cut to reformat input. For example:
line="5aaecdab287c90c50da70455de03fd1e ./2015/01/26/GOPR0083.MP4"
echo "$line" |
cut -d' ' -f2- |
xargs stat -c %.6Y

Unix file pattern issue: append changing value of variable pattern to copies of matching line

I have a file with contents:
abc|r=1,f=2,c=2
abc|r=1,f=2,c=2;r=3,f=4,c=8
I want a result like below:
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|3
The third column value is r value. A new line would be inserted for each occurrence.
I have tried with:
for i in `cat $xxxx.txt`
do
#echo $i
live=$(echo $i | awk -F " " '{print $1}')
home=$(echo $i | awk -F " " '{print $2}')
echo $live
done
but is not working properly. I am a beginner to sed/awk and not sure how can I use them. Can someone please help on this?
awk to the rescue!
$ awk -F'[,;|]' '{c=0;
for(i=2;i<=NF;i++)
if(match($i,/^r=/)) a[c++]=substr($i,RSTART+2);
delim=substr($0,length($0))=="|"?"":"|";
for(i=0;i<c;i++) print $0 delim a[i]}' file
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|3
Use an inner routine (made up of GNU grep, sed, and tr) to compile a second more elaborate sed command, the output of which needs further cleanup with more sed. Call the input file "foo".
sed -n $(grep -no 'r=[0-9]*' foo | \
sed 's/^[0-9]*/&s#.*#\&/;s/:r=/|/;s/.*/&#p;/' | \
tr -d '\n') foo | \
sed 's/|[0-9|]*|/|/'
Output:
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|3
Looking at the inner sed code:
grep -no 'r=[0-9]*' foo | \
sed 's/^[0-9]*/&s#.*#\&/;s/:r=/|/;s/.*/&#p;/' | \
tr -d '\n'
It's purpose is to parse foo on-the-fly (when foo changes, so will the output), and in this instance come up with:
1s#.*#&|1#p;2s#.*#&|1#p;2s#.*#&|3#p;
Which is almost perfect, but it leaves in old data on the last line:
sed -n '1s#.*#&|1#p;2s#.*#&|1#p;2s#.*#&|3#p;' foo
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1|3
...which old data |1 is what the final sed 's/|[0-9|]*|/|/' removes.
Here is a pure bash solution. I wouldn't recommend actually using this, but it might help you understand better how to work with files in bash.
# Iterate over each line, splitting into three fields
# using | as the delimiter. (f3 is only there to make
# sure a trailing | is not included in the value of f2)
while IFS="|" read -r f1 f2 f3; do
# Create an array of variable groups from $f2, using ;
# as the delimiter
IFS=";" read -a groups <<< "$f2"
for group in "${groups[#]}"; do
# Get each variable from the group separately
# by splitting on ,
IFS=, read -a vars <<< "$group"
for var in "${vars[#]}"; do
# Split each assignment on =, create
# the variable for real, and quit once we
# have found r
IFS== read name value <<< "$var"
declare "$name=$value"
[[ $name == r ]] && break
done
# Output the desired line for the current value of r
printf '%s|%s|%s\n' "$f1" "$f2" "$r"
done
done < $xxxx.txt
Changes for ksh:
read -A instead of read -a.
typeset instead of declare.
If <<< is a problem, you can use a here document instead. For example:
IFS=";" read -A groups <<EOF
$f2
EOF

Shell sed command

I have paths.txt like:
pathO1/:pathD1/
pathO2/:pathD2/
...
pathON/:pathDN/
How can I 'sed' insert ' * ' after each pathOX/ ?
The script is:
while read line
do
cp $(echo $line | tr ':' ' ')
done < "paths.txt"
substituted by:
while read line
do
cp $(echo $line | sed 's/:/* /1')
done < "paths.txt"
This looks to be a similar question to which you asked earlier: Shell Script: Read line in file
Just apply the trick of removing additional '*' before appliying tr like:
cp $(echo $line | sed 's/\*//1' | tr ':' '* ')
while read line
do
path=`echo "$line" | sed 's/:/ /g'`
cmd="cp $path"
echo $cmd
eval $cmd
done < "./paths.txt"
quick and dirty awk one-liner without loop to do the job:
awk -F: '$1="cp "$1' paths.txt
this will output:
cp /home/Documents/shellscripts/Origen/* /home/Documents/shellscripts/Destino/
cp /home/Documents/shellscripts/Origen2/* /home/Documents/shellscripts/Destino2/
...
if you want the cmds to get executed:
awk -F: '$1="cp "$1' paths.txt|sh
I said it quick & dirty, because:
the format must be path1:path2
your path cannot contain special letters (like space) or :
Using pure shell
while IFS=: read -r p1 p2
do
cp $p1 "$p2"
done < file

Transpose one line/lines from column to row using shell

I want convert a column of data in a txt file to a row of a csv file using unix commands.
example:
ApplChk1,
ApplChk2,
v_baseLoanAmountTI,
v_plannedClosingDateField,
downPaymentTI,
this is a column which present in a txt file
I want output as follows in a csv file
ApplChk1,ApplChk2,v_baseLoanAmountTI,v_plannedClosingDateField,downPaymentTI,
Please let me know how to do it.
Thanks in advance
If that's a single column, which you want to convert to row, then there are many possibilities:
tr -d '\n' < filename ; echo # option 1 OR
xargs echo -n < filename ; echo # option 2 (This option however, will shrink spaces & eat quotes) OR
while read x; do echo -n "$x" ; done < filename; echo # option 3
Please let us know, how the input would look like, for multi-line case.
A funny pure bash solution (bash ≥ 4.1):
mapfile -t < file.txt; printf '%s' "${MAPFILE[#]}" $'\n'
Done!
for i in `< file.txt` ; do echo -n $i; done; echo ""
gives the output
ApplChk1,ApplChk2,v_baseLoanAmountTI,v_plannedClosingDateField,downPaymentTI,
To send output to a file:
{ for i in `< file.txt` ; do echo -n $i ; done; echo; } > out.csv
When I run it, this is what happens:
[jenny#jennys:tmp]$ more file.txt
ApplChk1,
ApplChk2,
v_baseLoanAmountTI,
v_plannedClosingDateField,
downPaymentTI,
[jenny#jenny:tmp]$ { for i in `< file.txt` ; do echo -n $i ; done; echo; } > out.csv
[jenny#jenny:tmp]$ more out.csv
ApplChk1,ApplChk2,v_baseLoanAmountTI,v_plannedClosingDateField,downPaymentTI,
perl -pe 's/\n//g' your_file
the above will output to stdout.
if you want to do it in place:
perl -pi -e 's/\n//g' your_file
You could use the Linux command sed to replace line \n breaks by commas , or space :
sed -z 's/\n/,/g' test.txt > test.csv
You could also add the -i option if you want to change file in-place :
sed -i -z 's/\n/,/g' test.txt

Resources