Add/subtract variables in a really dumb shell - shell

I am writing a shell script which works on my local /bin/sh fine (dash on Ubuntu 13.04), but I unltimately need to run it on a dumb box where I'm getting an error because of an operation on variables:
$((n2 - n1 + 1))
doesn't work, I get an error like:
syntax error: you disabled math support for $((arith)) syntax
I don't know a lot about the sh on there but I think this thing is busybox. How can I do maths on this dumb shell?
edit with list of applets
~ # busybox --list
[
arp
ash
cat
chgrp
chmod
chown
chroot
chvt
clear
cmp
cp
cut
date
dd
deallocvt
df
dmesg
du
echo
env
false
find
freeramdisk
ftpget
ftpput
grep
gunzip
gzip
hexdump
hwclock
ifconfig
ln
losetup
ls
md5sum
mkdir
mkfifo
mknod
mkswap
more
mount
mv
nslookup
ping
ping6
ps
pwd
renice
reset
rm
rmdir
route
seq
sh
sha1sum
sha256sum
sleep
sort
swapoff
swapon
switch_root
sync
tar
taskset
tee
telnet
test
tftp
time
top
touch
true
umount
uname
uniq
uptime
usleep
vconfig
vi
wget
whoami
yes

Generic addition/subtraction/multiplication/division with seq+grep+sort
Notes:
All of these are POSIX-compliant, but there is a slightly faster non-POSIX subtract_nonposix which relies on a grep supporting -w and -B (non-POSIX, but even busybox' grep supports them)
add/subtract support only unsigned integers as input
multiply/divide support signed integers as input
subtract/multiply/divide can deal with negative results
depending on the input multiply/divide might be very costly (see comments)
subtract/multiply may pollute your namespace (they use $__x and $__y respectively) if not used in a subshell
arith.sh:
#!/bin/sh
is_uint()
{
case "$1" in
''|*[!0-9]*) return 1
;;
esac
[ "$1" -ge 0 ]
}
is_int()
{
case "${1#-}" in
''|*[!0-9]*) return 1
;;
esac
}
# requires seq, grep -n, sort -nr
# reasonably fast
add()
{
if ! is_uint "$1" \
|| ! is_uint "$2"; then
echo "Usage: add <uint1> <uint2>"
return 1
fi
[ "$1" -eq 0 ] && { echo "$2"; return; }
[ "$2" -eq 0 ] && { echo "$1"; return; }
{
seq 1 "$1"
seq 1 "$2"
} \
| grep -n "" \
| sort -nr \
| { read num; echo "${num%[-:]*}"; }
}
# requires seq, grep -n, sort -nr, uniq -u
# reasonably fast
subtract()
{
if ! is_uint "$1" \
|| ! is_uint "$2"; then
echo "Usage: subtract <uint1> <uint2>"
return 1
fi
if [ "$1" -ge "$2" ]; then
__x="$1"
__y="$2"
else
__x="$2"
__y="$1"
fi
{
seq 0 "${__x}"
seq 0 "${__y}"
} \
| sort -n \
| uniq -u \
| grep -n "" \
| sort -nr \
| \
{
read num
: ${num:=0}
[ "${__x}" = "$2" ] && [ "$1" -ne "$2" ] && minus='-'
echo "${minus}${num%:*}"
}
}
# requires seq, grep -wB
# faster than subtract(), but requires non-standard grep -wB
subtract_nonposix()
{
if ! is_uint "$1" \
|| ! is_uint "$2"; then
echo "Usage: subtract <uint1> <uint2>"
return 1
fi
if [ "$1" -ge "$2" ]; then
__x="$1"
__y="$2"
else
__x="$2"
__y="$1"
fi
seq 0 "${__x}" \
| grep -w -B "${__y}" "${__x}" \
| \
{
read num
[ "${__x}" = "$2" ] && [ "$1" -ne "$2" ] && minus='-'
echo "${minus}${num}"
}
}
# requires seq, sort -nr, add()
# very slow if multiplicand or multiplier is large
multiply()
{
if ! is_int "$1" \
|| ! is_int "$2"; then
echo "Usage: multiply <int1> <int2>"
return 1
fi
[ "$2" -eq 0 ] && { echo 0; return; }
# make sure to use the smaller number for the outer loop
# to speed up things a little if possible
if [ $1 -ge $2 ]; then
__x="$1"
__y="$2"
else
__x="$2"
__y="$1"
fi
__x="${__x#-}"
__y="${__y#-}"
seq 1 "${__y}" \
| while read num; do
sum="$(add "${sum:-0}" "${__x}")"
echo "${sum}"
done \
| sort -nr \
| \
{
read num
if [ "$1" -lt 0 -a "$2" -gt 0 ] \
|| [ "$2" -lt 0 -a "$1" -gt 0 ]; then
minus='-'
fi
echo "${minus}${num}"
}
}
# requires subtract()
# very costly if dividend is large and divisor is small
divide()
{
if ! is_int "$1" \
|| ! is_int "$2"; then
echo "Usage: divide <int1> <int2>"
return 1
fi
[ "$2" -eq 0 ] && { echo "division by zero"; return 1; }
(
sum="${1#-}"
y="${2#-}"
count=
while [ "${sum}" -ge "${y}" ]; do
sum="$(subtract "${sum}" "${y}")"
# no need to use add() for a simple +1 counter,
# this is way faster
count="${count}."
done
if [ "$1" -lt 0 -a "$2" -gt 0 ] \
|| [ "$2" -lt 0 -a "$1" -gt 0 ]; then
minus='-'
fi
echo "${minus}${#count}"
)
}
echo "10 4 14
4 10
10 10
2 -2
-2 -2
0 0
x y" | while read x y; do
for op in add subtract subtract_nonposix multiply divide; do
printf -- "${x} ${y} %-17s = %s\n" "${op}" "$("${op}" "${x}" "${y}")"
done
echo
done
Example run:
$ ./arith.sh
10 4 add = 14
10 4 subtract = 6
10 4 subtract_nonposix = 6
10 4 multiply = 40
10 4 divide = 2
4 10 add = 14
4 10 subtract = -6
4 10 subtract_nonposix = -6
4 10 multiply = 40
4 10 divide = 0
10 10 add = 20
10 10 subtract = 0
10 10 subtract_nonposix = 0
10 10 multiply = 100
10 10 divide = 1
2 -2 add = Usage: add <uint1> <uint2>
2 -2 subtract = Usage: subtract <uint1> <uint2>
2 -2 subtract_nonposix = Usage: subtract <uint1> <uint2>
2 -2 multiply = -4
2 -2 divide = -1
-2 -2 add = Usage: add <uint1> <uint2>
-2 -2 subtract = Usage: subtract <uint1> <uint2>
-2 -2 subtract_nonposix = Usage: subtract <uint1> <uint2>
-2 -2 multiply = 4
-2 -2 divide = 1
0 0 add = 0
0 0 subtract = 0
0 0 subtract_nonposix = 0
0 0 multiply = 0
0 0 divide = division by zero
x y add = Usage: add <uint1> <uint2>
x y subtract = Usage: subtract <uint1> <uint2>
x y subtract_nonposix = Usage: subtract <uint1> <uint2>
x y multiply = Usage: multiply <int1> <int2>
x y divide = Usage: divide <int1> <int2>

Another specific solution to your problem (n2 - n1 + 1) based on seq, sort -nr and uniq -u (POSIX-compliant).
foo()
{
{
seq 1 "$2"
seq 0 "$1"
} \
| sort -n \
| uniq -u \
| grep -n "" \
| sort -nr \
| { read num; echo "${num%:*}"; }
}
$ foo 100 2000
1901

head, tail and wc
If your busybox has head, tail and wc built in, you might try the following:
head -c $n2 /dev/zero | tail -c +$n1 | wc -c
The first will generate a sequence of n2 zero bytes. The second will start at position n1, counting from 1, so it will skip n1 - 1 bytes. Therefore the resulting sequence has n2 - n1 + 1 bytes. This count can be computed using wc -c.
head, tail and ls or stat
Tried this with my busybox, although its configuration might differ from yours. I'm not sure whether wc will be that more likely than expr. If you have head and tail but no wc, then you could probably write the result to a temporary file, and then use stat or ls to obtain the size as a string. Examples for this are included below.
seq and wc
If you have wc but not head and tail, then you could substitute seq instead:
seq $n1 $n2 | wc -l
seq, tr and stat
As your comment indicates you have no wc but do have seq, here is an alternative provided you have sufficuently complete ls and also tr, perhaps even stat. Alas, I just noticed that tr isn't in your list of applets either. Nevertheless, for future reference, here it is:
seq $n1 $n2 | tr -d [0-9] > tempfilename
stat -c%s tempfilename
This creates a sequence of n2 - n1 + 1 lines, then removes all digits, leaving only that many newlines, which it writes to a file. We then print its size.
dd and ls
But as you don't have tr, you'll need something different. dd might suite your needs, since you can use it a bit like head or tail.
dd if=/dev/zero of=tmp1 bs=1 count=$n2 # n2
dd if=tmp1 of=tmp2 bs=1 skip=$n1 # - n1
echo >> tmp2 # + 1
set -- dummy `ls -l tmp2`
echo $6
rm tmp1 tmp2
This creates a sequence of n2 null bytes, then skips the first n1 of it. It appends a single newline to add 1 to its size. Then it uses ls to print the size of that file, and sets the positional variables $1, $2, … based on its output. $6 should be the column containing the size. Unless I missed something, this should all be available to you.
Alternative to busybox
If everything else fails, you might still implement your own digit-wise subtraction algorithm, using a lot of case distinctions. But that would require a lot of work, so you might be better of shipping a statically linked expr binary, or something specifically designed for your use case, instead of a scripted approach.

Really weird idea - usable only if you have network connection:
a=2,3
b=2.7
res=`wget -q -O - "http://someyourserver:6000/($a+$b)*5/2"`
echo $res
so you can do calculations over the network. You must setup one simple web server will get the PATH_INFO from the request and return only the result.
the server part (very simplified - without any error handling etc.) can be like next app.psgi:
my $app = sub {
my $env = shift;
my $calc = $env->{PATH_INFO};
$calc =~ s:^/::; #remove 1.st slash
$calc =~ s:[^\d\(\)\+\*/\-\.\,]::g; #cleanup, only digits and +-*/()., allowed
$calc =~ s/,/\./g; #change , to .
my $res = eval $calc;
return [ 200, ['Content-Type' => 'text/plain'], [ "$res" ] ];
};
run with plackup -p 6000 app.psgi
or can use any other simple CGI or php script.

Alternatively, if you can reconfigure and rebuild BusyBox and enable "bash-compatible extensions", which should give you the ability to do mathematics. You will have to cross-compile your BusyBox again and replace the old binaries with your new one on your target (assuming you have the environment to do so). The BusyBox executable is only one binary, so you will only need to deal with simple replacement of one file.
I have BusyBox 1.19.4 and the mathematics evaluation works just fine.

Add/Subtract numbers using only printf
For me, the previous answers didn't work since I don't have seq, nor grep, nor wc, head or tail, not even dd.
My bash syntax doesn't support the math syntax $((n1+n2)), and not even the range syntax {1..N}. so it definitely was a tough environment.
I did manage to have basic add/subtract operations with small numbers (up to few thousands) using the following technique (calculate n1-n2):
n1=100
n2=20
str_n1=`printf "%${n1}s"` # "prints" 100 spaces, and store in str_n1
str_n2=`printf "%${n2}s"` # "prints" 20 spaces, and store in str_n2
if [ n1 -gt n2 ] # if the n1 > n2, then:
then
str_sub=${str_n1%$str_n2} #delete str_n2 from str_n1
else
str_sub=${str_n2%$str_n1} #delete str_n1 from str_n2
fi
# until now we created a string with 100 spaces, then a second string with 20 spaces, then we deleted the 20 of 2nd string from 1st string, so now all we left is to:
sub_result=${#str_sub} #check the length of str_sub
Same technique can be used also for adding numbers (continue from last example):
str_add=$str_n1$str_n2 # concat the two string to have 120 spaces together
add_result=${#str_add} # check the length of add_result
Now, in my case, I had to work with bigger numbers (up to ten of millions), and it cannot work with this method like that since it actually needs to print millions of spaces, and it takes forever.
Instead, since I don't need to whole number, but just a part of it, i took the middle of the number using the substring syntax:
n1=10058000
n2=10010000
n1=${n1:3:3} # -> 580 (takes 3 chars from the 3rd char)
n2=${n2:3:3} # -> 100
Then calculate what I need with smaller numbers (of course needs to take more parameters in consideration for cases like n1=10158000, and n2=10092000)

Here is the original solution I posted to your problem (n2 - n1 + 1) based on seq and grep.
foo()
{
seq 0 "$2" \
| grep -nw -B "$1" "$2" \
| { read num; echo "${num%[-:]*}"; }
}
$ foo 100 2000
1901
How it works:
First we generate a sequence of numbers from 0 to n2
Then we grep for n2 and include the leading n1 lines in the output. The first line holds our result now. We add the line number so the zero-based sequence accounts for the +1 (the line numbers and actual numbers will be off-by-one)
Then we fetch the first line with read (basically emulating head -n 1) and
discard the actual number from the output - the line number is the proper result

Related

how can I generate random numbers in specific range in bash using $RANDOM? [duplicate]

I need to generate a random port number between 2000-65000 from a shell script. The problem is $RANDOM is a 15-bit number, so I'm stuck!
PORT=$(($RANDOM%63000+2001)) would work nicely if it wasn't for the size limitation.
Does anyone have an example of how I can do this, maybe by extracting something from /dev/urandom and getting it within a range?
shuf -i 2000-65000 -n 1
Enjoy!
Edit: The range is inclusive.
On Mac OS X and FreeBSD you may also use jot:
jot -r 1 2000 65000
According to the bash man page, $RANDOM is distributed between 0 and 32767; that is, it is an unsigned 15-bit value. Assuming $RANDOM is uniformly distributed, you can create a uniformly-distributed unsigned 30-bit integer as follows:
$(((RANDOM<<15)|RANDOM))
Since your range is not a power of 2, a simple modulo operation will only almost give you a uniform distribution, but with a 30-bit input range and a less-than-16-bit output range, as you have in your case, this should really be close enough:
PORT=$(( ((RANDOM<<15)|RANDOM) % 63001 + 2000 ))
and here's one with Python
randport=$(python -S -c "import random; print random.randrange(2000,63000)")
and one with awk
awk 'BEGIN{srand();print int(rand()*(63000-2000))+2000 }'
The simplest general way that comes to mind is a perl one-liner:
perl -e 'print int(rand(65000-2000)) + 2000'
You could always just use two numbers:
PORT=$(($RANDOM + ($RANDOM % 2) * 32768))
You still have to clip to your range. It's not a general n-bit random number method, but it'll work for your case, and it's all inside bash.
If you want to be really cute and read from /dev/urandom, you could do this:
od -A n -N 2 -t u2 /dev/urandom
That'll read two bytes and print them as an unsigned int; you still have to do your clipping.
If you're not a bash expert and were looking to get this into a variable in a Linux-based bash script, try this:
VAR=$(shuf -i 200-700 -n 1)
That gets you the range of 200 to 700 into $VAR, inclusive.
Here's another one. I thought it would work on just about anything, but sort's random option isn't available on my centos box at work.
seq 2000 65000 | sort -R | head -n 1
Same with ruby:
echo $(ruby -e 'puts rand(20..65)') #=> 65 (inclusive ending)
echo $(ruby -e 'puts rand(20...65)') #=> 37 (exclusive ending)
Bash documentation says that every time $RANDOM is referenced, a random number between 0 and 32767 is returned. If we sum two consecutive references, we get values from 0 to 65534, which covers the desired range of 63001 possibilities for a random number between 2000 and 65000.
To adjust it to the exact range, we use the sum modulo 63001, which will give us a value from 0 to 63000. This in turn just needs an increment by 2000 to provide the desired random number, between 2000 and 65000. This can be summarized as follows:
port=$((((RANDOM + RANDOM) % 63001) + 2000))
Testing
# Generate random numbers and print the lowest and greatest found
test-random-max-min() {
max=2000
min=65000
for i in {1..10000}; do
port=$((((RANDOM + RANDOM) % 63001) + 2000))
echo -en "\r$port"
[[ "$port" -gt "$max" ]] && max="$port"
[[ "$port" -lt "$min" ]] && min="$port"
done
echo -e "\rMax: $max, min: $min"
}
# Sample output
# Max: 64990, min: 2002
# Max: 65000, min: 2004
# Max: 64970, min: 2000
Correctness of the calculation
Here is a full, brute-force test for the correctness of the calculation. This program just tries to generate all 63001 different possibilities randomly, using the calculation under test. The --jobs parameter should make it run faster, but it's not deterministic (total of possibilities generated may be lower than 63001).
test-all() {
start=$(date +%s)
find_start=$(date +%s)
total=0; ports=(); i=0
rm -f ports/ports.* ports.*
mkdir -p ports
while [[ "$total" -lt "$2" && "$all_found" != "yes" ]]; do
port=$((((RANDOM + RANDOM) % 63001) + 2000)); i=$((i+1))
if [[ -z "${ports[port]}" ]]; then
ports["$port"]="$port"
total=$((total + 1))
if [[ $((total % 1000)) == 0 ]]; then
echo -en "Elapsed time: $(($(date +%s) - find_start))s \t"
echo -e "Found: $port \t\t Total: $total\tIteration: $i"
find_start=$(date +%s)
fi
fi
done
all_found="yes"
echo "Job $1 finished after $i iterations in $(($(date +%s) - start))s."
out="ports.$1.txt"
[[ "$1" != "0" ]] && out="ports/$out"
echo "${ports[#]}" > "$out"
}
say-total() {
generated_ports=$(cat "$#" | tr ' ' '\n' | \sed -E s/'^([0-9]{4})$'/'0\1'/)
echo "Total generated: $(echo "$generated_ports" | sort | uniq | wc -l)."
}
total-single() { say-total "ports.0.txt"; }
total-jobs() { say-total "ports/"*; }
all_found="no"
[[ "$1" != "--jobs" ]] && test-all 0 63001 && total-single && exit
for i in {1..1000}; do test-all "$i" 40000 & sleep 1; done && wait && total-jobs
For determining how many iterations are needed to get a given probability p/q of all 63001 possibilities having been generated, I believe we can use the expression below. For example, here is the calculation for a probability greater than 1/2, and here for greater than 9/10.
$RANDOM is a number between 0 and 32767. You want a port between 2000 and 65000. These are 63001 possible ports. If we stick to values of $RANDOM + 2000 between 2000 and 33500, we cover a range of 31501 ports. If we flip a coin and then conditionally add 31501 to the result, we can get more ports, from 33501 to 65001. Then if we just drop 65001, we get the exact coverage needed, with a uniform probability distribution for all ports, it seems.
random-port() {
while [[ not != found ]]; do
# 2000..33500
port=$((RANDOM + 2000))
while [[ $port -gt 33500 ]]; do
port=$((RANDOM + 2000))
done
# 2000..65001
[[ $((RANDOM % 2)) = 0 ]] && port=$((port + 31501))
# 2000..65000
[[ $port = 65001 ]] && continue
echo $port
break
done
}
Testing
i=0
while true; do
i=$((i + 1))
printf "\rIteration $i..."
printf "%05d\n" $(random-port) >> ports.txt
done
# Then later we check the distribution
sort ports.txt | uniq -c | sort -r
You can do this
cat /dev/urandom|od -N2 -An -i|awk -v f=2000 -v r=65000 '{printf "%i\n", f + r * $1 / 65536}'
If you need more details see Shell Script Random Number Generator.
PORT=$(($RANDOM%63000+2001)) is close to what you want I think.
PORT=$(($RANDOM$RANDOM$RANDOM%63000+2001)) gets around the size limitation that troubles you. Since bash makes no distinctions between a number variable and a string variable, this works perfectly well. The "number" $RANDOM can be concatenated like a string, and then used as a number in a calculation. Amazing!
Or on OS-X the following works for me:
$ gsort --random-sort
Generate random numbers in the range [$floor,$ceil), no dependence:
$(((RANDOM % $(($ceil- $floor))) + $floor))
Generate 100 numbers between 2000 to 65000:
for i in $(seq 100); do echo $(((RANDOM % $((65000 - 2000))) + 2000));done
You can get the random number through urandom
head -200 /dev/urandom | cksum
Output:
3310670062 52870
To retrieve the one part of the above number.
head -200 /dev/urandom | cksum | cut -f1 -d " "
Then the output is
3310670062
To meet your requirement,
head -200 /dev/urandom |cksum | cut -f1 -d " " | awk '{print $1%63000+2001}'
This is how I usually generate random numbers. Then I use "NUM_1" as the variable for the port number I use. Here is a short example script.
#!/bin/bash
clear
echo 'Choose how many digits you want for port# (1-5)'
read PORT
NUM_1="$(tr -dc '0-9' </dev/urandom | head -c $PORT)"
echo "$NUM_1"
if [ "$PORT" -gt "5" ]
then
clear
echo -e "\x1b[31m Choose a number between 1 and 5! \x1b[0m"
sleep 3
clear
exit 0
fi
This works for me:
export CUDA_VISIBLE_DEVICES=$((( RANDOM % 8 )))
you can add 1 if you want it to start from 1 instead of 0.
Generating 50 numbers in Bash from a range 100000000000-999999999999 and saving them into a file filename.csv
shuf -i 100000000000-999999999999 -n 50 -o filename.csv
If you need a range bigger than 15 bit, dont use the slow unsecure and outdated 15 bit RANDOM, use the fast and secure 32 bit SRANDOM.
SRANDOM are available since about 2021 bash 5.1 roll out.
"one interesting addition to note with Bash 5.1 is the new SRANDOM variable. The SRANDOM variable provides random data from the system's entropy engine and cannot be reseeded. In particular, the SRANDOM variable provides a 32-bit random number that relies upon getrandom/getentropy -- with fall-backs to /dev/urandom or arc4random or even another fallback after that if necessary."
Source: https://www.phoronix.com/news/GNU-Bash-5.1
See what are the different of RANDOM and SRANDOM in bash:
Difference between RANDOM and SRANDOM in Bash
Feel free to improve this answer.

Use of associative array with a variable name under bash 4.1

I trying to parse multiples files like this under bash-4.1
$cat hostname_abc.txt
host_type type_foo
SoftA version123
SoftB version456
to obtain an output where you can see how many times a version of Soft[A,B] is used, grouped by host type :
$./list_versions.sh
[type_foo] 11 times
SoftA:
[version123] 1 times
[version444] 5 times
[version567] 5 times
SoftB:
[version456] 9 times
[version777] 2 times
[type_bar] 6 times
SoftA:
[version444] 6 times
SoftB:
[version111] 4 times
[version777] 2 times
I don't know in advance the list of host_type and the versions.
So I tried to save in an associative array the count of each host_type and create dynamically the names of the associatives arrays which stored the count of each version of Soft[A,B] per host_type based base on a template host_type_Soft[A,B]
I tried many times with different variations of syntax and indirections so I remade below a more readable script that follow my aim :
#!/usr/bin/env bash
# ----- generated test conditions -----
echo -e "host_type typeA\nSoftA v2\nSoftB v1" > hostname_1.txt
echo -e "host_type typeB\nSoftA v1\nSoftB v1" > hostname_2.txt
echo -e "host_type typeB\nSoftA v1\nSoftB v0" > hostname_3.txt
echo -e "host_type typeA\nSoftA v0\nSoftB v0" > hostname_4.txt
echo -e "host_type typeA\nSoftA v3\nSoftB v2" > hostname_5.txt
echo -e "host_type typeB\nSoftA v3\nSoftB v1" > hostname_6.txt
echo -e "host_type typeB\nSoftA v2\nSoftB v2" > hostname_7.txt
echo -e "host_type typeA\nSoftA v1\nSoftB v2" > hostname_8.txt
echo -e "host_type typeC\nSoftA v0\nSoftB v4" > hostname_9.txt
list_hostname() {
for i in {1..9}; do
echo "hostname_${i}.txt"
done
}
declare -A list_host_type
while read f; do
#parse the hostname files
while read l; do
[[ $l = *"host_type"* ]] && host_type="$( echo $l | cut -d' ' -f2)"
[[ $l = *"SoftA"* ]] && versionA="$( echo $l | cut -d' ' -f2)"
[[ $l = *"SoftB"* ]] && versionB="$( echo $l | cut -d' ' -f2)"
done < <( cat "$f" )
#count the number of hosts by host_type
[[ ${list_host_type[$host_type]} ]] && ((list_host_type[$host_type]++)) || list_host_type[$host_type]='1'
#create associative arrays with a name only know at runtime
declare -A "${host_type}_SoftA"
declare -A "${host_type}_SoftB"
#count the number of host for the couple host_type and Soft[A,B], stored on the dynamically named assiociative array
[[ ${${host_type}_SoftA[$versionA]} ]] && ((${host_type}_SoftA[$versionA]++)) || ${host_type}_SoftA[$versionA]='1'
[[ ${${host_type}_SoftB[$versionB]} ]] && ((${host_type}_SoftB[$versionB]++)) || ${host_type}_SoftB[$versionB]='1'
done < <( list_hostname )
#print a non pretty-formated output
echo '==== result ====='
for m in "${!list_host_type[#]}"; do
echo "host type: $m count: ${list_model[$m]}"
for versionA in "${!${m}_softA[#]}"; do
echo " SoftA version: $versionA count: ${${m}_SoftA[$versionA]}"
done
for versionB in "${!${m}_softB[#]}"; do
echo " SoftB version: $versionB count: ${${m}_SoftB[$versionB]}"
done
done
I know they are others methods to achieve my goal but I want to know if I can use associative this way with bash-4.1.
I don't think you can use dynamic variable names with arrays in Bash.
(I tried a few things but couldn't figure out the syntax.)
Even if possible, I think it would be extremely difficult to understand.
A possible workaround could be using a single associative array,
with "composite keys".
That is, for example use a comma separated value of host type, soft and version:
while read f; do
line=0
while read col1 col2; do
if [[ $line = 0 ]]; then
host_type=$col2
else
soft=$col1
version=$col2
index=$host_type,$soft,$version
((list_host_type[$index]++))
fi
((line++))
done < <( cat "$f" )
done < <( list_hostname )
for m in "${!list_host_type[#]}"; do
echo $m = ${list_host_type[$m]}
done
For your sample data this would produce:
typeA,SoftA,v2 = 1
typeA,SoftA,v3 = 1
typeA,SoftA,v0 = 1
typeA,SoftA,v1 = 1
typeB,SoftA,v3 = 1
typeB,SoftA,v2 = 1
typeB,SoftA,v1 = 2
typeA,SoftB,v2 = 2
typeA,SoftB,v1 = 1
typeA,SoftB,v0 = 1
typeC,SoftB,v4 = 1
typeB,SoftB,v2 = 1
typeB,SoftB,v0 = 1
typeB,SoftB,v1 = 2
typeC,SoftA,v0 = 1
And then work with this associative array to compute the statistics you need. Here's a rough example implementation:
get_host_types() {
local names=(${!list_host_type[#]})
printf "%s\n" "${names[#]%%,*}" | sort -u
}
get_soft() {
local host_type=$1
local names=(${!list_host_type[#]})
for name in "${names[#]}"; do
[[ ${name%%,*} = $host_type ]] && echo $name
done | cut -d, -f2 | sort -u
}
get_versions() {
local prefix=$1
local names=(${!list_host_type[#]})
for name in "${names[#]}"; do
[[ ${name%,*} = $prefix ]] && echo $name
done | cut -d, -f3 | sort -u
}
indent=" "
for host_type in $(get_host_types); do
echo "[$host_type]"
for soft in $(get_soft $host_type); do
echo "$indent$soft:"
for version in $(get_versions $host_type,$soft); do
index=$host_type,$soft,$version
echo "$indent$indent[$version] ${list_host_type[$index]} times"
done
done
done
Producing as output:
[typeA]
SoftA:
[v0] 1 times
[v1] 1 times
[v2] 1 times
[v3] 1 times
SoftB:
[v0] 1 times
[v1] 1 times
[v2] 2 times
[typeB]
SoftA:
[v1] 2 times
[v2] 1 times
[v3] 1 times
SoftB:
[v0] 1 times
[v1] 2 times
[v2] 1 times
[typeC]
SoftA:
[v0] 1 times
SoftB:
[v4] 1 times
All in all, it would be better to implement this using a proper programming language.

Division in script and floating-point

I would like to do the following operation in my script:
1 - ((m - 20) / 34)
I would like to assign the result of this operation to another variable. I want my script use floating point math. For example, for m = 34:
results = 1 - ((34 - 20) / 34) == 0.588
You could use the bc calculator. It will do arbitrary precision math using decimals (not binary floating point) if you set increease scale from its default of 0:
$ m=34
$ bc <<< "scale = 10; 1 - (($m - 20) / 34)"
.5882352942
The -l option will load the standard math library and default the scale to 20:
$ bc -l <<< "1 - (($m - 20) / 34)"
.58823529411764705883
You can then use printf to format the output, if you so choose:
printf "%.3f\n" "$(bc -l ...)"
Bash does not do floating point math. You can use awk or bc to handle this. Here is an awk example:
$ m=34; awk -v m=$m 'BEGIN { print 1 - ((m - 20) / 34) }'
0.588235
To assign the output to a variable:
var=$(awk -v m=$m 'BEGIN { print 1 - ((m - 20) / 34) }')
Teach bash e.g. integer division with floating point results:
#!/bin/bash
div () # Arguments: dividend and divisor
{
if [ $2 -eq 0 ]; then echo division by 0; return 1; fi
local p=12 # precision
local c=${c:-0} # precision counter
local d=. # decimal separator
local r=$(($1/$2)); echo -n $r # result of division
local m=$(($r*$2))
[ $c -eq 0 ] && [ $m -ne $1 ] && echo -n $d
[ $1 -eq $m ] || [ $c -eq $p ] && echo && return
local e=$(($1-$m))
c=$(($c+1))
div $(($e*10)) $2
}
result=$(div 1080 633) # write to variable
echo $result
result=$(div 7 34)
echo $result
result=$(div 8 32)
echo $result
result=$(div 246891510 2)
echo $result
result=$(div 5000000 177)
echo $result
Output:
1.706161137440
0.205882352941
0.25
123445755
28248.587570621468
echo $a/$b|bc -l
gives the result.
Example:
read a b
echo $a/$b|bc -l
Enter a & b value as 10 3, you get 3.3333333333
If you want to store the value in another variable then use the code
read a b
c=`echo $a/$b|bc -l`
echo $c
It also gives the same result as above.
Try it...
I know this is an old thread, but this seemed like a fun project to tackle without using bc or invoking recursion. I'm sure it can be improved, but this maxed out my skill.
numerator=5
denominator=7 # - 0 -> returns "undef"
decimal_places=4 # - 0 -> same as echo $(( $numerator / $denominator ))
_result_sign=""
let _dp_exp=10**decimal_places
if [ $denominator -eq 0 ]; then _div_result_int_large=0; else let _div_result_int_large=$((numerator * _dp_exp / denominator)); fi
if [ $_div_result_int_large -lt 0 ]; then let _div_result_int_large=$(( _div_result_int_large * -1 )); _result_sign="-"; fi
let _div_result_int=$((_div_result_int_large / _dp_exp))
let _div_result_mant=$((_div_result_int_large - _div_result_int * _dp_exp))
let _dp_lzeros=$((decimal_places - ${#_div_result_mant}))
printf -v _div_result_mant_padded "%.${_dp_lzeros}d$_div_result_mant"
div_result="$_result_sign$_div_result_int"
if [ $decimal_places -gt 0 ]; then div_result="$_result_sign$_div_result_int.$_div_result_mant_padded"; fi
if [ $denominator -eq 0 ]; then div_result="undef"; fi
echo $div_result
Example output:
numerator=5
denominator=7
decimal_places=5
-> 0.71428
numerator=250
denominator=13
decimal_places=0
-> 19
numerator=-5
denominator=6
decimal_places=2
-> -0.83
numerator=3
denominator=0 # - uh-oh
decimal_places=2 # - can be anything, in this case
-> undef
Use this script open this file with favorite editor like:
$ sudo vim /usr/bin/div
Then paste this code:
#!/bin/bash
# Author: Danial Rikhteh Garan (danial.rikhtehgaran#gmail.com)
if [[ -z "$1" ]] || [[ -z "$2" ]]; then
echo "Please input two number"
echo "for 100/50 use: div 10 50"
exit 1;
fi
div=$(echo "$1/$2" | bc -l);
echo 0$div | sed 's/[0]*$//g'
Now chmod it to 755:
$ sudo chmod 755 /usr/bin/div
Now use it:
$ div 5 100
0.05
In your script you can use this:
var=$(div 5 100);
echo "$var"

bashscript for changing wput uploaddirectory on every 5000 passage

how i can change the uploaddirectory of wput on every 5000 passage?
whats wrong in my code?
only for the record; i need infnite loop i want run this script 24h 365 days!
every 5000 passage one number higher in directory like incoming1-infinite
best regards
haug
#!/bin/bash
for (( ; ; ))
do
no=1
while [ $no -le 5000 ]
do
perl job.pl false false 1
cd comp
for fname in *.jpg;
do
mv -i "$fname" ${RANDOM}${RANDOM}.jpg
done
for fname in *.jpg;
do
mv "$fname" $(echo "$fname" | sha1sum | cut -f1 -d' ').jpg
done
if [ $no = '5000' ]
then
echo $no
echo $n
n=$(( n+1 )) # <-- this one not do what i need :-(
else
wput *.jpg ftp://user:pass#ip:port/incoming$n/ # <-- here is the important part 2, i need it to change one directory higher every 5000 passage for example; incoming1 until infinite
rm -rf *.jpg
cd ..
fi
no=`expr $no + 1`
done
done
cd comp
for ((no=1; ; no++))
do
n=`expr $no / 5000`
perl job.pl false false 1
for fname in *.jpg;
# ... rest of the script
done
Just calculate the remainder of division by 5000 and if it is equal 0 (or 1 or anything < 5000) do what you need.
if [ $((no%5000)) = 0 ]
then
...
fi
Just put a for-loop that executes 5,000 times inside your infinite loop (which can keep track of the directory sequence number for you):
#!/bin/bash
for (( n=0; ; n++ )); do
for (( no=1; no < 5000; no++ )); do
perl job.pl false false 1
( # Subshell so we don't have to worry about returning
# to the previous directory.
cd comp
# No need to rename each file twice.
for fname in *.jpg; do
mv "$fname" $(echo "${RANDOM}${RANDOM}.jpg" | sha1sum | cut -f1 -d' ').jpg
done
wput *.jpg ftp://user:pass#ip:port/incoming$n
)
done
done

Random number from a range in a Bash Script

I need to generate a random port number between 2000-65000 from a shell script. The problem is $RANDOM is a 15-bit number, so I'm stuck!
PORT=$(($RANDOM%63000+2001)) would work nicely if it wasn't for the size limitation.
Does anyone have an example of how I can do this, maybe by extracting something from /dev/urandom and getting it within a range?
shuf -i 2000-65000 -n 1
Enjoy!
Edit: The range is inclusive.
On Mac OS X and FreeBSD you may also use jot:
jot -r 1 2000 65000
According to the bash man page, $RANDOM is distributed between 0 and 32767; that is, it is an unsigned 15-bit value. Assuming $RANDOM is uniformly distributed, you can create a uniformly-distributed unsigned 30-bit integer as follows:
$(((RANDOM<<15)|RANDOM))
Since your range is not a power of 2, a simple modulo operation will only almost give you a uniform distribution, but with a 30-bit input range and a less-than-16-bit output range, as you have in your case, this should really be close enough:
PORT=$(( ((RANDOM<<15)|RANDOM) % 63001 + 2000 ))
and here's one with Python
randport=$(python -S -c "import random; print random.randrange(2000,63000)")
and one with awk
awk 'BEGIN{srand();print int(rand()*(63000-2000))+2000 }'
The simplest general way that comes to mind is a perl one-liner:
perl -e 'print int(rand(65000-2000)) + 2000'
You could always just use two numbers:
PORT=$(($RANDOM + ($RANDOM % 2) * 32768))
You still have to clip to your range. It's not a general n-bit random number method, but it'll work for your case, and it's all inside bash.
If you want to be really cute and read from /dev/urandom, you could do this:
od -A n -N 2 -t u2 /dev/urandom
That'll read two bytes and print them as an unsigned int; you still have to do your clipping.
If you're not a bash expert and were looking to get this into a variable in a Linux-based bash script, try this:
VAR=$(shuf -i 200-700 -n 1)
That gets you the range of 200 to 700 into $VAR, inclusive.
Here's another one. I thought it would work on just about anything, but sort's random option isn't available on my centos box at work.
seq 2000 65000 | sort -R | head -n 1
Same with ruby:
echo $(ruby -e 'puts rand(20..65)') #=> 65 (inclusive ending)
echo $(ruby -e 'puts rand(20...65)') #=> 37 (exclusive ending)
Bash documentation says that every time $RANDOM is referenced, a random number between 0 and 32767 is returned. If we sum two consecutive references, we get values from 0 to 65534, which covers the desired range of 63001 possibilities for a random number between 2000 and 65000.
To adjust it to the exact range, we use the sum modulo 63001, which will give us a value from 0 to 63000. This in turn just needs an increment by 2000 to provide the desired random number, between 2000 and 65000. This can be summarized as follows:
port=$((((RANDOM + RANDOM) % 63001) + 2000))
Testing
# Generate random numbers and print the lowest and greatest found
test-random-max-min() {
max=2000
min=65000
for i in {1..10000}; do
port=$((((RANDOM + RANDOM) % 63001) + 2000))
echo -en "\r$port"
[[ "$port" -gt "$max" ]] && max="$port"
[[ "$port" -lt "$min" ]] && min="$port"
done
echo -e "\rMax: $max, min: $min"
}
# Sample output
# Max: 64990, min: 2002
# Max: 65000, min: 2004
# Max: 64970, min: 2000
Correctness of the calculation
Here is a full, brute-force test for the correctness of the calculation. This program just tries to generate all 63001 different possibilities randomly, using the calculation under test. The --jobs parameter should make it run faster, but it's not deterministic (total of possibilities generated may be lower than 63001).
test-all() {
start=$(date +%s)
find_start=$(date +%s)
total=0; ports=(); i=0
rm -f ports/ports.* ports.*
mkdir -p ports
while [[ "$total" -lt "$2" && "$all_found" != "yes" ]]; do
port=$((((RANDOM + RANDOM) % 63001) + 2000)); i=$((i+1))
if [[ -z "${ports[port]}" ]]; then
ports["$port"]="$port"
total=$((total + 1))
if [[ $((total % 1000)) == 0 ]]; then
echo -en "Elapsed time: $(($(date +%s) - find_start))s \t"
echo -e "Found: $port \t\t Total: $total\tIteration: $i"
find_start=$(date +%s)
fi
fi
done
all_found="yes"
echo "Job $1 finished after $i iterations in $(($(date +%s) - start))s."
out="ports.$1.txt"
[[ "$1" != "0" ]] && out="ports/$out"
echo "${ports[#]}" > "$out"
}
say-total() {
generated_ports=$(cat "$#" | tr ' ' '\n' | \sed -E s/'^([0-9]{4})$'/'0\1'/)
echo "Total generated: $(echo "$generated_ports" | sort | uniq | wc -l)."
}
total-single() { say-total "ports.0.txt"; }
total-jobs() { say-total "ports/"*; }
all_found="no"
[[ "$1" != "--jobs" ]] && test-all 0 63001 && total-single && exit
for i in {1..1000}; do test-all "$i" 40000 & sleep 1; done && wait && total-jobs
For determining how many iterations are needed to get a given probability p/q of all 63001 possibilities having been generated, I believe we can use the expression below. For example, here is the calculation for a probability greater than 1/2, and here for greater than 9/10.
$RANDOM is a number between 0 and 32767. You want a port between 2000 and 65000. These are 63001 possible ports. If we stick to values of $RANDOM + 2000 between 2000 and 33500, we cover a range of 31501 ports. If we flip a coin and then conditionally add 31501 to the result, we can get more ports, from 33501 to 65001. Then if we just drop 65001, we get the exact coverage needed, with a uniform probability distribution for all ports, it seems.
random-port() {
while [[ not != found ]]; do
# 2000..33500
port=$((RANDOM + 2000))
while [[ $port -gt 33500 ]]; do
port=$((RANDOM + 2000))
done
# 2000..65001
[[ $((RANDOM % 2)) = 0 ]] && port=$((port + 31501))
# 2000..65000
[[ $port = 65001 ]] && continue
echo $port
break
done
}
Testing
i=0
while true; do
i=$((i + 1))
printf "\rIteration $i..."
printf "%05d\n" $(random-port) >> ports.txt
done
# Then later we check the distribution
sort ports.txt | uniq -c | sort -r
You can do this
cat /dev/urandom|od -N2 -An -i|awk -v f=2000 -v r=65000 '{printf "%i\n", f + r * $1 / 65536}'
If you need more details see Shell Script Random Number Generator.
PORT=$(($RANDOM%63000+2001)) is close to what you want I think.
PORT=$(($RANDOM$RANDOM$RANDOM%63000+2001)) gets around the size limitation that troubles you. Since bash makes no distinctions between a number variable and a string variable, this works perfectly well. The "number" $RANDOM can be concatenated like a string, and then used as a number in a calculation. Amazing!
Or on OS-X the following works for me:
$ gsort --random-sort
Generate random numbers in the range [$floor,$ceil), no dependence:
$(((RANDOM % $(($ceil- $floor))) + $floor))
Generate 100 numbers between 2000 to 65000:
for i in $(seq 100); do echo $(((RANDOM % $((65000 - 2000))) + 2000));done
You can get the random number through urandom
head -200 /dev/urandom | cksum
Output:
3310670062 52870
To retrieve the one part of the above number.
head -200 /dev/urandom | cksum | cut -f1 -d " "
Then the output is
3310670062
To meet your requirement,
head -200 /dev/urandom |cksum | cut -f1 -d " " | awk '{print $1%63000+2001}'
This is how I usually generate random numbers. Then I use "NUM_1" as the variable for the port number I use. Here is a short example script.
#!/bin/bash
clear
echo 'Choose how many digits you want for port# (1-5)'
read PORT
NUM_1="$(tr -dc '0-9' </dev/urandom | head -c $PORT)"
echo "$NUM_1"
if [ "$PORT" -gt "5" ]
then
clear
echo -e "\x1b[31m Choose a number between 1 and 5! \x1b[0m"
sleep 3
clear
exit 0
fi
This works for me:
export CUDA_VISIBLE_DEVICES=$((( RANDOM % 8 )))
you can add 1 if you want it to start from 1 instead of 0.
Generating 50 numbers in Bash from a range 100000000000-999999999999 and saving them into a file filename.csv
shuf -i 100000000000-999999999999 -n 50 -o filename.csv
If you need a range bigger than 15 bit, dont use the slow unsecure and outdated 15 bit RANDOM, use the fast and secure 32 bit SRANDOM.
SRANDOM are available since about 2021 bash 5.1 roll out.
"one interesting addition to note with Bash 5.1 is the new SRANDOM variable. The SRANDOM variable provides random data from the system's entropy engine and cannot be reseeded. In particular, the SRANDOM variable provides a 32-bit random number that relies upon getrandom/getentropy -- with fall-backs to /dev/urandom or arc4random or even another fallback after that if necessary."
Source: https://www.phoronix.com/news/GNU-Bash-5.1
See what are the different of RANDOM and SRANDOM in bash:
Difference between RANDOM and SRANDOM in Bash
Feel free to improve this answer.

Resources