Why use seq 0 in bash for loop?
for i in `seq 0 $(( ${#ARRAYEX[#]} - 1 ))`
do
echo "ARRAYEX${i}=${ARRAYEX[${i}]}"
done
The seq command generates a sequence of numbers.
For example
seq 0 10
generates a sequence of numbers from 0 up to 10:
0 1 2 3 4 5 6 7 8 9 10
(usually each number is on a new line, but I place them after each other)
In your example a sequence on number starting at 0 up to the size of the array minus 1 is generated.
The seq 0 $(( ${#ARRAYEX[#]} - 1 )) part expands to:
0 1 2 3 4
assuming that the ARRAYEX has a size of 5.
Inside the loop the array is used again, so the loop is iterating over all array element (as the first element of the array starts at 0).
seq 0 $(( ${#ARRAYEX[#]} - 1 )) creates a sequence of all the possible indexes of the array. You can also use
for ((i=0; i<${#ARRAYEX[#]}; ++i )) ; do
Related
I'm trying to make the coordinate "x" randomly move in the interval [-1,1]. However, my code works sometimes, and sometimes it doesn't. I tried ShellCheck but it says "no issues detected!". I'm new to conditionals, am I using them wrong?
I'm running this on the windows subsystem for linux. I'm editing it on nano. Since I have a script that will plot 200 of these "random walks", the code should work consistenly, but I really don't understant why it doesn't.
Here's my code:
x=0
for num in {1..15}
do
r=$RANDOM
if [[ $r -lt 16383 ]]
then
p=1
else
p=-1
fi
if [[ $x -eq $p ]]
then
x=$(echo "$x-$p" | bc )
else
x=$(echo "$x+$p" | bc )
fi
echo "$num $x"
done
I expect something like this:
1 -1
2 0
3 1
4 0
5 1
6 0
7 1
8 0
9 1
10 0
11 -1
12 0
13 1
14 0
15 1
But the usual output is something like this:
1 1
2 0
3 -1
4 0
5 -1
6 0
7 -1
(standard_in) 1: syntax error
8
(standard_in) 1: syntax error
9
(standard_in) 1: syntax error
10
(standard_in) 1: syntax error
11
(standard_in) 1: syntax error
12
(standard_in) 1: syntax error
13
(standard_in) 1: syntax error
14
(standard_in) 1: syntax error
15
Always stopping after a -1.
You can do this with bash:
x=$(( x - p ))
or
(( x -= p ))
and you don't need bc.
Replace x=$(echo "$x-$p" | bc ) with x=$(echo "$x-($p)" | bc ) to avoid echo "-1--1" | bc.
One-liner equivalents to the OP's 18-line random walk script, using bash arithmetic evaluation:
x=0; printf '%-5s\n' {1..15}\ $(( x=(RANDOM%2 ? 1 : -1) * (x==0) ))
x=0; printf '%-5s\n' {1..15}\ $(( x=( x ? 0 : (RANDOM%2 ? 1 : -1) ) ))
Sample output of either, (the 2nd column will vary between runs):
1 -1
2 0
3 -1
4 0
5 1
6 0
7 1
8 0
9 -1
10 0
11 1
12 0
13 -1
14 0
15 -1
How it works:
echo {1..15}\ $(( ...some code... )) prints the numbers 1 to
15 followed by 15 instances of whatever result in the $(( ... )) code returns. One flaw with this approach is that with the resulting 15 pairs of numbers, (e.g. 1 -1, 2 0, etc.), each appears to bash as one string, rather than 30 separate numbers.
(RANDOM%2): the % is a modulo operator and here returns the remainder when divided by 2, which is either 0 or 1.
(x==0): $x can be one of three numbers, but if the previous value of $x was -1 or 1 the only legal random step is 0, so we only need a random number if the previous value of $x was 0.
The if logic is replaced with shortcuts of the form (expr?expr:expr); these use the same logic as the OP script.
My data(tab separated):
1 0 0 1 0 1 1 0 1
1 1 0 1 0 1 0 1 1
1 1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0 0
...
how can i grep the lines with exact, for example, 5 '1's,
ideal output:
1 0 0 1 0 1 1 0 1
Also, how can i grep lines with equal or more than (>=) 5 '1's,
ideal output:
1 0 0 1 0 1 1 0 1
1 1 0 1 0 1 0 1 1
1 1 1 1 1 1 1 1 1
i tried,
grep 1$'\t'1$'\t'1$'\t'1$'\t'1
however this will only output consecutive '1's, which is not all i want.
i wonder if there will be any simple method to achieve this, thank you!
John Bollinger's helpful answer and anishane's answer show that it can be done with grep, but, as has been noted, that is quite cumbersome, given that regular expression aren't designed for counting.
awk, by contrast, is built for field-based parsing and counting (often combined with regular expressions to identify field separators, or, as below, the fields themselves).
Assuming you have GNU awk, you can use the following:
Exactly 5 1s:
awk -v FPAT='\\<1\\>' 'NF==5' file
5 or more 1s:
awk -v FPAT='\\<1\\>' 'NF>=5' file
Special variable FPAT is a GNU awk extension that allows you to identify fields via a regex that describes the fields themselves, in contrast with the standard approach of using a regex to define the separators between fields (via special variable FS or option -F):
'\\<1\\>' identifies any "isolated" 1 (surrounded by non-word characters) as a field, based on word-boundary assertions \< and \>; the \ must be doubled here so that the initial string parsing performed by awk doesn't "eat" single \s.
Standard variable NF contains the count of input fields in the line at hand, which allows easy numerical comparison. If the conditional evaluates to true, the input line at hand is implicitly printed (in other words: NF==5 is implicitly the same as NF==5 { print } and, more verbosely, NF==5 { print $0 }).
A POSIX-compliant awk solution is a little more complicated:
Exactly 5 1s:
awk '{ l=$0; gsub("[\t0]", "") }; length($0)==5 { print l }' file
5 or more 1s:
awk '{ l=$0; gsub("[\t0]", "") }; length($0)>=5 { print l }' file
l=$0 saves the input line ($0) in its original form in variable l.
gsub("[\t0]", "") replaces all \t and 0 chars. in the input line with the empty string, i.e., effectively removes them, and only leaves (directly concatenated) 1 instances (if any).
length($0)==5 { print l } then prints the original input line (l) only if the resulting string of 1s (i.e., the count of 1s now stored in the modified input line ($0)) matches the specified count.
You can use grep. But that would be an abuse of regex.
$ cat countme
1 0 0 1 0 1 1 0 1
1 1 0 1 0 1 0 1 1
1 1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0 0
$ grep -P '^[0\t]*(1[0\t]*){5}[0\t]*$' countme # Match exactly 5
1 0 0 1 0 1 1 0 1
$ grep -P '^[0\t]*(1[0\t]*){5,}[0\t]*$' countme # Match >=5
1 0 0 1 0 1 1 0 1
1 1 0 1 0 1 0 1 1
1 1 1 1 1 1 1 1 1
You can do this to get lines with exactly five '1's:
grep '^[^1]*\(1[^1]*\)\{5,5\}[^1]*$'
You can simplify that to this for at least five '1's:
grep '\(1[^1]*\)\{5,\}'
The enumerated quantifier (\{n,m\}) enables you to conveniently specify a particular number or range of numbers of consecutive matches to a sub-pattern. To avoid matching lines with extra matches to such a pattern, however, you must also anchor it to the beginning and end of the line.
The other other trick is to make sure the gaps previous to the first 1, between the 1s, and after the last 1 are matched. In your case, all of those gaps can be represented pretty simply as ranges of zero or more characters other than 1: [^1]*. Putting those pieces together gives you the above regular expressions.
Do
sed -nE '/^([^1]*1[^1]*){5}$/p' your_file
for exactly 5 matches and
sed -nE '/^([^1]*1[^1]*){5,}$/p' your_file
for 5 or more matches.
Note: In GNU sed you may not see the -E option in the manpage, but it is supported. Using -E is for portability to, say, Mac OSX.
with perl
$ perl -ane 'print if (grep {$_==1} #F) == 5' ip.txt
1 0 0 1 0 1 1 0 1
$ perl -ane 'print if (grep {$_==1} #F) >= 5' ip.txt
1 0 0 1 0 1 1 0 1
1 1 0 1 0 1 0 1 1
1 1 1 1 1 1 1 1 1
-a to automatically split input line on whitespaces and save to #F array
grep {$_==1} #F returns array with elements from #F array which are exactly equal to 1
(grep {$_==1} #F) == 5 in scalar context, comparison will be done based on number of elements of array
See http://perldoc.perl.org/perlrun.html#Command-Switches for details on -ane options
I have been googling and trying different methods but nothing seems to work.
I have the following code
string=0 4 5 27 8 7 0 6
total=0
for n in "$string"; do
total=$(($total + $n))
done
This way I want to count the total sum of all the numbers within that string.
I have also tried expr "$total" + "$n" but that gives me an error saying the operand is not an integer.
Any suggestion how I might make this work?
Don't quote the string in the in clause, quoted string is not split into words:
#! /bin/bash
total=0
string='0 4 5 27 8 7 0 6'
for n in $string ; do
(( total += n ))
done
echo $total
string=0 4 5 27 8 7 0 6
This attempts to set the variable string to 0, then invoke the command 4 with arguments 5 27 8 7 0 6.
You need to quote the value:
string="0 4 5 27 8 7 0 6"
And you need to remove the quotes when you refer to it; change
for n in "$string"; do
to
for n in $string; do
You should use :
total=$(( total + n ))
no need for the $ before variables inside a $(( )) statement
I have the following shell script that reads in data from a file inputted at the command line. The file is a matrix of numbers, and I need to separate the file by columns and then sort the columns. Right now I can read the file and output the individual columns but I am getting lost on how to sort. I have inputted a sort statement, but it only sorts the first column.
EDIT:
I have decided to take another route and actual transpose the matrix to turn the columns into rows. Since I have to later calculate the mean and median and have already successfully done this for the file row-wise earlier in the script - it was suggested to me to try and "spin" the matrix if you will to turn the columns into rows.
Here is my UPDATED code
declare -a col=( )
read -a line < "$1"
numCols=${#line[#]} # save number of columns
index=0
while read -a line ; do
for (( colCount=0; colCount<${#line[#]}; colCount++ )); do
col[$index]=${line[$colCount]}
((index++))
done
done < "$1"
for (( width = 0; width < numCols; width++ )); do
for (( colCount = width; colCount < ${#col[#]}; colCount += numCols ) ); do
printf "%s\t" ${col[$colCount]}
done
printf "\n"
done
This gives me the following output:
1 9 6 3 3 6
1 3 7 6 4 4
1 4 8 8 2 4
1 5 9 9 1 7
1 5 7 1 4 7
Though I'm now looking for:
1 3 3 6 6 9
1 3 4 4 6 7
1 2 4 4 8 8
1 1 5 7 9 9
1 1 4 5 7 7
To try and sort the data, I have tried the following:
sortCol=${col[$colCount]}
eval col[$colCount]='($(sort <<<"${'$sortCol'[*]}"))'
Also: (which is how I sorted the row after reading in from line)
sortCol=( $(printf '%s\t' "${col[$colCount]}" | sort -n) )
If you could provide any insight on this, it would be greatly appreciated!
Note, as mentioned in the comments, a pure bash solution isn't pretty. There are a number of ways to do it, but this is probably the most straight forward. The following requires reading all values per line into the array, and saving the matrix stride so it can be transposed to read all column values into a row matrix and sorted. All sorted columns are inserted into new row matrix a2. Transposing that row matrix yields your original matrix back in column sort order.
Note this will work for any rank of column matrix in your file.
#!/bin/bash
test -z "$1" && { ## validate number of input
printf "insufficient input. usage: %s <filename>\n" "${0//*\//}"
exit 1;
}
test -r "$1" || { ## validate file was readable
printf "error: file not readable '%s'. usage: %s <filename>\n" "$1" "${0//*\//}"
exit 1;
}
## function: my sort integer array - accepts array and returns sorted array
## Usage: array=( "$(msia ${array[#]})" )
msia() {
local a=( "$#" )
local sz=${#a[#]}
local _tmp
[[ $sz -lt 2 ]] && { echo "Warning: array not passed to fxn 'msia'"; return 1; }
for((i=0;i<$sz;i++)); do
for((j=$((sz-1));j>i;j--)); do
[[ ${a[$i]} -gt ${a[$j]} ]] && {
_tmp=${a[$i]}
a[$i]=${a[$j]}
a[$j]=$_tmp
}
done
done
echo ${a[#]}
unset _tmp
unset sz
return 0
}
declare -a a1 ## declare arrays and matrix variables
declare -a a2
declare -i cnt=0
declare -i stride=0
declare -i sz=0
while read line; do ## read all lines into array
a1+=( $line );
(( cnt == 0 )) && stride=${#a1[#]} ## calculate matrix stride
(( cnt++ ))
done < "$1"
sz=${#a1[#]} ## calculate matrix size
## print original array
printf "\noriginal array:\n\n"
for ((i = 0; i < sz; i += stride)); do
for ((j = 0; j < stride; j++)); do
printf " %s" ${a1[i+j]}
done
printf "\n"
done
## sort columns from stride array
for ((j = 0; j < stride; j++)); do
for ((i = 0; i < sz; i += stride)); do
arow+=( ${a1[i+j]} )
done
a2+=( $(msia ${arow[#]}) ) ## create sorted array
unset arow
done
## print the sorted array
printf "\nsorted array:\n\n"
for ((j = 0; j < cnt; j++)); do
for ((i = 0; i < sz; i += cnt)); do
printf " %s" ${a2[i+j]}
done
printf "\n"
done
exit 0
Output
$ bash sort_cols2.sh dat/matrix.txt
original array:
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
sorted array:
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
Awk script
awk '
{for(i=1;i<=NF;i++)a[i]=a[i]" "$i} #Add to column array
END{
for(i=1;i<=NF;i++){
split(a[i],b) #Split column
x=asort(b) #sort column
for(j=1;j<=x;j++){ #loop through sort
d[j]=d[j](d[j]~/./?" ":"")b[j] #Recreate lines
}
}
for(i=1;i<=NR;i++)print d[i] #Print lines
}' file
Output
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
Here's my entry in this little exercise. Should handle an arbitrary number of columns. I assume they're space-separated:
#!/bin/bash
linenumber=0
while read line; do
i=0
# Create an array for each column.
for number in $line; do
[ $linenumber == 0 ] && eval "array$i=()"
eval "array$i+=($number)"
(( i++ ))
done
(( linenumber++ ))
done <$1
IFS=$'\n'
# Sort each column
for j in $(seq 0 $i ); do
thisarray=array$j
eval array$j='($(sort <<<"${'$thisarray'[*]}"))'
done
# Print each array's 0'th entry, then 1, then 2, etc...
for k in $(seq 0 ${#array0[#]}); do
for j in $(seq 0 $i ); do
eval 'printf ${array'$j'['$k']}" "'
done
echo ""
done
Not bash but i think this python code worths a look showing how this task can be achieved using built-in functions.
From the interpreter:
$ cat matrix.txt
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
$ python
Python 2.7.3 (default, Jun 19 2012, 17:11:17)
[GCC 4.4.3] on hp-ux11
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> f = open('./matrix.txt')
>>> for row in zip(*[sorted(list(a))
for a in zip(*[a.split() for a in f.readlines()])]):
... print ' '.join(row)
...
1 1 1 1 1
3 3 2 1 1
3 4 4 5 4
6 4 4 7 5
6 6 8 9 7
9 7 8 9 7
I am new to shell scripting and I am trying a simple task of getting the length of a sequence of numbers generated using seq.
With the help of a related post here: How to find the array length in unix shell? I was able to do this -
a=(1 2 3 4 5)
echo ${#a[#]} #length of a
5 #length of a = 5 (This is fine !!)
However when I try to do a similar thing using seq ..
b=$(seq 1 1 10)
echo $b
1 2 3 4 5 6 7 8 9 10
echo ${#b[#]}
1 #the length of b is 1, while I expect it to be 10
Why does this happen ? Are the variable types a and b different? is b not an array ?
I am sure I am missing something very trivial here, help is greatly appreciated.
Thanks
Ashwin
You need to store the output in an array to find the length of the array:
$ b=($(seq 1 1 10))
$ echo ${#b[#]}
10
Saying b=$(seq 1 1 10) doesn't produce an array.
Try
echo ${b[0]}
It will be 1 2 3 4 5 6 7 8 9 10 because all your values are stored in first element of array a as a string.
b=($(seq 1 1 10))
will do what you want.