I have a nested for loop to print one letter each from each variable.
for i in a b ; do for j in 1 2; do echo "$i $j"; done; done
a 1
a 2
b 1
b 2
My requirement is to have as
a 1
b 2
How do I get it ?
letters=(a b c d) # declare an array with four elements
numbers=(1 2 3 4)
for ((i=0;i<${#letters[#]};i++)); do echo ${letters[$i]} ${numbers[$i]}; done
Output:
a 1
b 2
c 3
d 4
${#letters[#]} is the number of elements in array letters.
You can also do the same using regular variables and string indexes:
#!/bin/bash
letters="abcdefghi"
nums="123456789"
for ((i = 0; i < ${#nums}; i++)); do
printf "%s %s\n" ${letters:i:1} ${nums:i:1}
done
Output
$ bash prnidx.sh
a 1
b 2
c 3
d 4
e 5
f 6
g 7
h 8
i 9
Related
I'm using a particular program that would require me to examine pairs of variables in a text file by specifying the pairs using indices.
For example:
gcta --reml-bivar 1 2 --grm test --pheno test.phen --out test
Where 1 and 2 would correspond to values from the first two columns in a text file. If I had 50 columns and wanted to examine each pair without repetition (1&2, 2&3, 1&3 ... 50), what would be the best way to automate this by looping through this? So essentially the script would be executing the same command but taking in pairs of indices like:
gcta --reml-bivar 1 3 --grm test --pheno test.phen --out test
gcta --reml-bivar 1 4 --grm test --pheno test.phen --out test
... so on and so forth. Thanks!
Since you haven't shown us any sample input we're just guessing but if your input is list of numbers (extracted from a file or otherwise) then here's an approach:
$ cat combinations.awk
###################
# Calculate all combinations of a set of strings, see
# https://rosettacode.org/wiki/Combinations#AWK
###################
function get_combs(A,B, i,n,comb) {
## Default value for r is to choose 2 from pool of all elements in A.
## Can alternatively be set on the command line:-
## awk -v r=<number of items being chosen> -f <scriptname>
n = length(A)
if (r=="") r = 2
comb = ""
for (i=1; i <= r; i++) { ## First combination of items:
indices[i] = i
comb = (i>1 ? comb OFS : "") A[indices[i]]
}
B[comb]
## While 1st item is less than its maximum permitted value...
while (indices[1] < n - r + 1) {
## loop backwards through all items in the previous
## combination of items until an item is found that is
## less than its maximum permitted value:
for (i = r; i >= 1; i--) {
## If the equivalently positioned item in the
## previous combination of items is less than its
## maximum permitted value...
if (indices[i] < n - r + i) {
## increment the current item by 1:
indices[i]++
## Save the current position-index for use
## outside this "for" loop:
p = i
break
}
}
## Put consecutive numbers in the remainder of the array,
## counting up from position-index p.
for (i = p + 1; i <= r; i++) indices[i] = indices[i - 1] + 1
## Print the current combination of items:
comb = ""
for (i=1; i <= r; i++) {
comb = (i>1 ? comb OFS : "") A[indices[i]]
}
B[comb]
}
}
# Input should be a list of strings
{
split($0,A)
delete B
get_combs(A,B)
PROCINFO["sorted_in"] = "#ind_str_asc"
for (comb in B) {
print comb
}
}
.
$ awk -f combinations.awk <<< '1 2 3 4'
1 2
1 3
1 4
2 3
2 4
3 4
.
$ while read -r a b; do
echo gcta --reml-bivar "$a" "$b" --grm test --pheno test.phen --out test
done < <(awk -f combinations.awk <<< '1 2 3 4')
gcta --reml-bivar 1 2 --grm test --pheno test.phen --out test
gcta --reml-bivar 1 3 --grm test --pheno test.phen --out test
gcta --reml-bivar 1 4 --grm test --pheno test.phen --out test
gcta --reml-bivar 2 3 --grm test --pheno test.phen --out test
gcta --reml-bivar 2 4 --grm test --pheno test.phen --out test
gcta --reml-bivar 3 4 --grm test --pheno test.phen --out test
Remove the echo when you're done testing and happy with the output.
In case anyone's reading this and wants permutations instead of combinations:
$ cat permutations.awk
###################
# Calculate all permutations of a set of strings, see
# https://en.wikipedia.org/wiki/Heap%27s_algorithm
function get_perm(A, i, lgth, sep, str) {
lgth = length(A)
for (i=1; i<=lgth; i++) {
str = str sep A[i]
sep = " "
}
return str
}
function swap(A, x, y, tmp) {
tmp = A[x]
A[x] = A[y]
A[y] = tmp
}
function generate(n, A, B, i) {
if (n == 1) {
B[get_perm(A)]
}
else {
for (i=1; i <= n; i++) {
generate(n - 1, A, B)
if ((n%2) == 0) {
swap(A, 1, n)
}
else {
swap(A, i, n)
}
}
}
}
function get_perms(A,B) {
generate(length(A), A, B)
}
###################
# Input should be a list of strings
{
split($0,A)
delete B
get_perms(A,B)
PROCINFO["sorted_in"] = "#ind_str_asc"
for (perm in B) {
print perm
}
}
.
$ awk -f permutations.awk <<< '1 2 3 4'
1 2 3 4
1 2 4 3
1 3 2 4
1 3 4 2
1 4 2 3
1 4 3 2
2 1 3 4
2 1 4 3
2 3 1 4
2 3 4 1
2 4 1 3
2 4 3 1
3 1 2 4
3 1 4 2
3 2 1 4
3 2 4 1
3 4 1 2
3 4 2 1
4 1 2 3
4 1 3 2
4 2 1 3
4 2 3 1
4 3 1 2
4 3 2 1
Both of the above use GNU awk for sorted_in to sort the output. If you don't have GNU awk you can still use the scripts as-is and if you need to sort the output then pipe it to sort.
If I understand you correctly and you don't need pairs looks like '1 1', '2 2', ... and '1 2', '2 1' ... try this script
#!/bin/bash
for i in $(seq 1 49);
do
for j in $(seq $(($i + 1)) 50);
do gcta --reml-bivar "$i $j" --grm test --pheno test.phen --out test
done;
done;
1 and 2 would correspond to values from the first two columns in a text file.
each pair without repetition
So let's walk through this process:
We repeat the first column from the file times the file length
We repeat each value (each line) from the second column from the file times the file length
We join the repeated columns -> we have all combinations
We need to filter "repetitions", we can just join the file with the original file and filter out repeating columns
So we get each pair without repetitions.
Then we just read the file line by line.
The script:
# create an input file cause you didn't provide any
cat << EOF > in.txt
1 a
2 b
3 c
4 d
EOF
# get file length
inlen=$(<in.txt wc -l)
# join the columns
paste -d' ' <(
# repeat the first column inlen times
# https://askubuntu.com/questions/521465/how-can-i-repeat-the-content-of-a-file-n-times
seq "$inlen" |
xargs -I{} cut -d' ' -f1 in.txt
) <(
# repeat each line inlen times
# https://unix.stackexchange.com/questions/81904/repeat-each-line-multiple-times
awk -v IFS=' ' -v v="$inlen" '{for(i=0;i<v;i++)print $2}' in.txt
) |
# filter out repetitions - ie. filter original lines from the file
sort |
comm --output-delimiter='' -3 <(sort in.txt) - |
# read the file line by line
while read -r one two; do
echo "$one" "$two"
done
will output:
1 b
1 c
1 d
2 a
2 c
2 d
3 a
3 b
3 d
4 a
4 b
4 c
#!/bin/bash
#set the length of the combination depending the
#user's choice
eval rg+=({1..$2})
#the code builds the script and runs it (eval)
eval `
#Character range depending on user selection
for i in ${rg[#]} ; do
echo "for c$i in {1..$1} ;do "
done ;
#Since the script is based on a code that brings
#all possible combinations even with duplicates -
#this is where the deduplication
#prevention conditioning set by (the script writes
#the conditioning code)
op1=$2
op2=$(( $2 - 1 ))
echo -n "if [ 1 == 1 ] "
while [ $op1 -gt 1 ] ; do
echo -n \&\& [ '$c'$op1 != '$c'$op2 ]' '
op2=$(( op2 -1 )
if [ $op2 == 0 ] ; then
op1=$(( op1 - 1 ))
op2=$(( op1 - 1 ))
fi
done ;
echo ' ; then'
echo -n "echo "
for i in ${rg[#]} ;
do
echo -n '$c'$i
done ;
echo \;
echo fi\;
for i in ${rg[#]} ; do
echo 'done ;'
done;`
example: range length
$ ./combs.bash '{1..2} {a..c} \$ \#' 4
12ab$
12ab#
12acb
12ac$
12ac#
12a$b
12a$c
12a$#
12a#b
12a#c
12a#$
..........
#!/bin/bash
len=$2
eval c=($1)
per()
{
((`grep -Poi '[^" ".]'<<<$2|sort|uniq|wc -l` < $((len - ${1}))))&&{ return;}
(($1 == 0))&&{ echo $2;return;}
for i in ${c[#]} ; do
per "$((${1} - 1 ))" "$2 $i"
done
}
per "$2" ""
#example
$ ./neto '{0..3} {a..d} \# \!' 7
0 1 2 3 a b c
0 1 2 3 a b d
0 1 2 3 a b #
0 1 2 3 a b !
0 1 2 3 a c b
0 1 2 3 a c d
0 1 2 3 a c #
0 1 2 3 a c !
0 1 2 3 a d b
...
I try made a restore point from 9F4A but i have no idea how did it
punto_de_restauracion="9F4A"
a=9
b=F
c=4
d=A
for a in 4 A 5 9 E 3 F 6 C 2 7
do
for b in 4 A 5 9 E 3 F 6 C 2 7
do
for c in 4 A 5 9 E 3 F 6 C 2 7
do
for d in 4 A 5 9 E 3 F 6 C 2 7
do
echo "$a$b$c$d"
done
done
done
done
I try restart a for loop, start from -> 9F4A and increment
What I think you're asking is how to exit from all the nested do loops when a=9, b=F, c=4 and d=A. If so, you want the shell command "break":
#!/bin/sh
#punto_de_restauracion="9F4A"
for a in 4 A 5 9 E 3 F 6 C 2 7
do
for b in 4 A 5 9 E 3 F 6 C 2 7
do
for c in 4 A 5 9 E 3 F 6 C 2 7
do
for d in 4 A 5 9 E 3 F 6 C 2 7
do
echo "$a$b$c$d"
if [ "$a" == "9" -a "$b" == "F" -a "$c" == "4" -a "$d" == "A" ] ; then
break 4
fi
done
done
done
done
You can get more information on the "break" command in the "SHELL BUILTIN COMMANDS" section of the "sh" man page.
The easy thing to do is set a flag when the point from which you want to restart is reached, and avoid actually taking actions inside your loop until that flag is set. Using continue skips the inner loops, so you don't bother iterating through values of b, c or d when you haven't yet found the right a (for example).
restart_point="9F4A"
if [[ $restart_point ]]; then
started=0
else
started=1
fi
not_started() {
(( started == 1 )) && return 1 # if started is true, we're not not_started (false!)
[[ ${1:$2:1} != "$3" ]] && return 0 # otherwise, with no match, we're not_started (true!)
[[ $final ]] && started=1 # w/ match, if "final" is set, set "started" flag
return 1 # since we found a match earlier, we're true.
}
for a in 4 A 5 9 E 3 F 6 C 2 7; do
not_started "$restart_point" 0 "$a" && continue
for b in 4 A 5 9 E 3 F 6 C 2 7; do
not_started "$restart_point" 1 "$b" && continue
for c in 4 A 5 9 E 3 F 6 C 2 7; do
not_started "$restart_point" 2 "$c" && continue
for d in 4 A 5 9 E 3 F 6 C 2 7; do
final=1 not_started "$restart_point" 3 "$d" && continue
echo "$a$b$c$d"
done
done
done
done
I have the following shell script that reads in data from a file inputted at the command line. The file is a matrix of numbers, and I need to separate the file by columns and then sort the columns. Right now I can read the file and output the individual columns but I am getting lost on how to sort. I have inputted a sort statement, but it only sorts the first column.
EDIT:
I have decided to take another route and actual transpose the matrix to turn the columns into rows. Since I have to later calculate the mean and median and have already successfully done this for the file row-wise earlier in the script - it was suggested to me to try and "spin" the matrix if you will to turn the columns into rows.
Here is my UPDATED code
declare -a col=( )
read -a line < "$1"
numCols=${#line[#]} # save number of columns
index=0
while read -a line ; do
for (( colCount=0; colCount<${#line[#]}; colCount++ )); do
col[$index]=${line[$colCount]}
((index++))
done
done < "$1"
for (( width = 0; width < numCols; width++ )); do
for (( colCount = width; colCount < ${#col[#]}; colCount += numCols ) ); do
printf "%s\t" ${col[$colCount]}
done
printf "\n"
done
This gives me the following output:
1 9 6 3 3 6
1 3 7 6 4 4
1 4 8 8 2 4
1 5 9 9 1 7
1 5 7 1 4 7
Though I'm now looking for:
1 3 3 6 6 9
1 3 4 4 6 7
1 2 4 4 8 8
1 1 5 7 9 9
1 1 4 5 7 7
To try and sort the data, I have tried the following:
sortCol=${col[$colCount]}
eval col[$colCount]='($(sort <<<"${'$sortCol'[*]}"))'
Also: (which is how I sorted the row after reading in from line)
sortCol=( $(printf '%s\t' "${col[$colCount]}" | sort -n) )
If you could provide any insight on this, it would be greatly appreciated!
Note, as mentioned in the comments, a pure bash solution isn't pretty. There are a number of ways to do it, but this is probably the most straight forward. The following requires reading all values per line into the array, and saving the matrix stride so it can be transposed to read all column values into a row matrix and sorted. All sorted columns are inserted into new row matrix a2. Transposing that row matrix yields your original matrix back in column sort order.
Note this will work for any rank of column matrix in your file.
#!/bin/bash
test -z "$1" && { ## validate number of input
printf "insufficient input. usage: %s <filename>\n" "${0//*\//}"
exit 1;
}
test -r "$1" || { ## validate file was readable
printf "error: file not readable '%s'. usage: %s <filename>\n" "$1" "${0//*\//}"
exit 1;
}
## function: my sort integer array - accepts array and returns sorted array
## Usage: array=( "$(msia ${array[#]})" )
msia() {
local a=( "$#" )
local sz=${#a[#]}
local _tmp
[[ $sz -lt 2 ]] && { echo "Warning: array not passed to fxn 'msia'"; return 1; }
for((i=0;i<$sz;i++)); do
for((j=$((sz-1));j>i;j--)); do
[[ ${a[$i]} -gt ${a[$j]} ]] && {
_tmp=${a[$i]}
a[$i]=${a[$j]}
a[$j]=$_tmp
}
done
done
echo ${a[#]}
unset _tmp
unset sz
return 0
}
declare -a a1 ## declare arrays and matrix variables
declare -a a2
declare -i cnt=0
declare -i stride=0
declare -i sz=0
while read line; do ## read all lines into array
a1+=( $line );
(( cnt == 0 )) && stride=${#a1[#]} ## calculate matrix stride
(( cnt++ ))
done < "$1"
sz=${#a1[#]} ## calculate matrix size
## print original array
printf "\noriginal array:\n\n"
for ((i = 0; i < sz; i += stride)); do
for ((j = 0; j < stride; j++)); do
printf " %s" ${a1[i+j]}
done
printf "\n"
done
## sort columns from stride array
for ((j = 0; j < stride; j++)); do
for ((i = 0; i < sz; i += stride)); do
arow+=( ${a1[i+j]} )
done
a2+=( $(msia ${arow[#]}) ) ## create sorted array
unset arow
done
## print the sorted array
printf "\nsorted array:\n\n"
for ((j = 0; j < cnt; j++)); do
for ((i = 0; i < sz; i += cnt)); do
printf " %s" ${a2[i+j]}
done
printf "\n"
done
exit 0
Output
$ bash sort_cols2.sh dat/matrix.txt
original array:
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
sorted array:
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
Awk script
awk '
{for(i=1;i<=NF;i++)a[i]=a[i]" "$i} #Add to column array
END{
for(i=1;i<=NF;i++){
split(a[i],b) #Split column
x=asort(b) #sort column
for(j=1;j<=x;j++){ #loop through sort
d[j]=d[j](d[j]~/./?" ":"")b[j] #Recreate lines
}
}
for(i=1;i<=NR;i++)print d[i] #Print lines
}' file
Output
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
Here's my entry in this little exercise. Should handle an arbitrary number of columns. I assume they're space-separated:
#!/bin/bash
linenumber=0
while read line; do
i=0
# Create an array for each column.
for number in $line; do
[ $linenumber == 0 ] && eval "array$i=()"
eval "array$i+=($number)"
(( i++ ))
done
(( linenumber++ ))
done <$1
IFS=$'\n'
# Sort each column
for j in $(seq 0 $i ); do
thisarray=array$j
eval array$j='($(sort <<<"${'$thisarray'[*]}"))'
done
# Print each array's 0'th entry, then 1, then 2, etc...
for k in $(seq 0 ${#array0[#]}); do
for j in $(seq 0 $i ); do
eval 'printf ${array'$j'['$k']}" "'
done
echo ""
done
Not bash but i think this python code worths a look showing how this task can be achieved using built-in functions.
From the interpreter:
$ cat matrix.txt
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
$ python
Python 2.7.3 (default, Jun 19 2012, 17:11:17)
[GCC 4.4.3] on hp-ux11
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> f = open('./matrix.txt')
>>> for row in zip(*[sorted(list(a))
for a in zip(*[a.split() for a in f.readlines()])]):
... print ' '.join(row)
...
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
It sounds simple but I can't do it in a simple way. In shell for loop, two vars
A=" 1 2 3 4"
B=" a b c d"
, try to print 1a 2b 3c 4d. Tried
A=" 1 2 3 4"
B=" a b c d"
for m in $A
for n in $B;
do echo $m$n done.
The output is
1
2
3
4
5
for
l
in
a
b
c
d
e
Anyone can help this out?
Here's one way to do it:
$ A=(1 2 3 4); B=(a b c d); for i in $(seq 0 3); do echo ${A[$i]}${B[$i]}; done
1a
2b
3c
4d
In your attempt, the for cases aren't closed with a ;, so it keeps interpreting words in your second for statement as cases for the first for statement.
Use instead:
A="1 2 3 4"
B="a b c d"
for m in $A ; do
for n in $B ; do
echo $m$n
done
done
I have file like this
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
* a
0 b
I want delete a, b from two last Records in END{} section
Result:
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
*
0
How can I get n last lines and change fields on them with awk?
Here's one way using any awk:
awk -v count=$(wc -l <file.txt) 'NR > count - 2 { $2 = "" }1' file.txt
Results:
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
*
0
Or to do awk operations for all records except 2 last lines of input file as a shell script, try ./script.sh file.txt. Contents of script.sh:
command=$(awk -v count=$(wc -l <"$1") 'NR <= count - 2 { $2 = "" }1' "$1"
echo -e "$command"
Results:
1 "45554323" p b
2 "34534567" f a
3 "76546787" u b
2 "56765435" f a
* a
0 b
If you know the value of n - the line number after which you want to delete the last item on the line/colum (here 4) this will work:
awk '{if (NR>4) NF=NF-1}1' data.txt
will give:
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
*
0
NF = NF -1 makes awk think there is one less field on the line than there is, which is how it doesn't display the last column/item on the line once that condition is met. NR refers to the current line number in the file being read.
awk can't know the number of lines in a file unless it goes through it once, or is given that information (e.g., wc -l). An alternative approach would be to save the last n lines in a buffer (sort of a sliding window/tape-delay type analogy, you are always printing n lines behind) and then process the final n lines in the END block.
This doesn't exactly answer your question but it produces the output you require:
$ gawk '{if (NF < 3) print $1; else print}' input.txt
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
*
0
$ cat file
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
* a
0 b
$ awk 'BEGIN{ARGV[ARGC++]=ARGV[ARGC-1]} NR==FNR{nr++; next} FNR>(nr-2) {NF--} 1' file
1 2 "45554323" p b
2 2 "34534567" f a
3 3 "76546787" u b
2 4 "56765435" f a
*
0
or if you don't mind manually specifying the file name twice:
awk 'NR==FNR{nr++; next} FNR>(nr-2) {NF--} 1' file file