Related
I need to generate a random port number between 2000-65000 from a shell script. The problem is $RANDOM is a 15-bit number, so I'm stuck!
PORT=$(($RANDOM%63000+2001)) would work nicely if it wasn't for the size limitation.
Does anyone have an example of how I can do this, maybe by extracting something from /dev/urandom and getting it within a range?
shuf -i 2000-65000 -n 1
Enjoy!
Edit: The range is inclusive.
On Mac OS X and FreeBSD you may also use jot:
jot -r 1 2000 65000
According to the bash man page, $RANDOM is distributed between 0 and 32767; that is, it is an unsigned 15-bit value. Assuming $RANDOM is uniformly distributed, you can create a uniformly-distributed unsigned 30-bit integer as follows:
$(((RANDOM<<15)|RANDOM))
Since your range is not a power of 2, a simple modulo operation will only almost give you a uniform distribution, but with a 30-bit input range and a less-than-16-bit output range, as you have in your case, this should really be close enough:
PORT=$(( ((RANDOM<<15)|RANDOM) % 63001 + 2000 ))
and here's one with Python
randport=$(python -S -c "import random; print random.randrange(2000,63000)")
and one with awk
awk 'BEGIN{srand();print int(rand()*(63000-2000))+2000 }'
The simplest general way that comes to mind is a perl one-liner:
perl -e 'print int(rand(65000-2000)) + 2000'
You could always just use two numbers:
PORT=$(($RANDOM + ($RANDOM % 2) * 32768))
You still have to clip to your range. It's not a general n-bit random number method, but it'll work for your case, and it's all inside bash.
If you want to be really cute and read from /dev/urandom, you could do this:
od -A n -N 2 -t u2 /dev/urandom
That'll read two bytes and print them as an unsigned int; you still have to do your clipping.
If you're not a bash expert and were looking to get this into a variable in a Linux-based bash script, try this:
VAR=$(shuf -i 200-700 -n 1)
That gets you the range of 200 to 700 into $VAR, inclusive.
Here's another one. I thought it would work on just about anything, but sort's random option isn't available on my centos box at work.
seq 2000 65000 | sort -R | head -n 1
Same with ruby:
echo $(ruby -e 'puts rand(20..65)') #=> 65 (inclusive ending)
echo $(ruby -e 'puts rand(20...65)') #=> 37 (exclusive ending)
Bash documentation says that every time $RANDOM is referenced, a random number between 0 and 32767 is returned. If we sum two consecutive references, we get values from 0 to 65534, which covers the desired range of 63001 possibilities for a random number between 2000 and 65000.
To adjust it to the exact range, we use the sum modulo 63001, which will give us a value from 0 to 63000. This in turn just needs an increment by 2000 to provide the desired random number, between 2000 and 65000. This can be summarized as follows:
port=$((((RANDOM + RANDOM) % 63001) + 2000))
Testing
# Generate random numbers and print the lowest and greatest found
test-random-max-min() {
max=2000
min=65000
for i in {1..10000}; do
port=$((((RANDOM + RANDOM) % 63001) + 2000))
echo -en "\r$port"
[[ "$port" -gt "$max" ]] && max="$port"
[[ "$port" -lt "$min" ]] && min="$port"
done
echo -e "\rMax: $max, min: $min"
}
# Sample output
# Max: 64990, min: 2002
# Max: 65000, min: 2004
# Max: 64970, min: 2000
Correctness of the calculation
Here is a full, brute-force test for the correctness of the calculation. This program just tries to generate all 63001 different possibilities randomly, using the calculation under test. The --jobs parameter should make it run faster, but it's not deterministic (total of possibilities generated may be lower than 63001).
test-all() {
start=$(date +%s)
find_start=$(date +%s)
total=0; ports=(); i=0
rm -f ports/ports.* ports.*
mkdir -p ports
while [[ "$total" -lt "$2" && "$all_found" != "yes" ]]; do
port=$((((RANDOM + RANDOM) % 63001) + 2000)); i=$((i+1))
if [[ -z "${ports[port]}" ]]; then
ports["$port"]="$port"
total=$((total + 1))
if [[ $((total % 1000)) == 0 ]]; then
echo -en "Elapsed time: $(($(date +%s) - find_start))s \t"
echo -e "Found: $port \t\t Total: $total\tIteration: $i"
find_start=$(date +%s)
fi
fi
done
all_found="yes"
echo "Job $1 finished after $i iterations in $(($(date +%s) - start))s."
out="ports.$1.txt"
[[ "$1" != "0" ]] && out="ports/$out"
echo "${ports[#]}" > "$out"
}
say-total() {
generated_ports=$(cat "$#" | tr ' ' '\n' | \sed -E s/'^([0-9]{4})$'/'0\1'/)
echo "Total generated: $(echo "$generated_ports" | sort | uniq | wc -l)."
}
total-single() { say-total "ports.0.txt"; }
total-jobs() { say-total "ports/"*; }
all_found="no"
[[ "$1" != "--jobs" ]] && test-all 0 63001 && total-single && exit
for i in {1..1000}; do test-all "$i" 40000 & sleep 1; done && wait && total-jobs
For determining how many iterations are needed to get a given probability p/q of all 63001 possibilities having been generated, I believe we can use the expression below. For example, here is the calculation for a probability greater than 1/2, and here for greater than 9/10.
$RANDOM is a number between 0 and 32767. You want a port between 2000 and 65000. These are 63001 possible ports. If we stick to values of $RANDOM + 2000 between 2000 and 33500, we cover a range of 31501 ports. If we flip a coin and then conditionally add 31501 to the result, we can get more ports, from 33501 to 65001. Then if we just drop 65001, we get the exact coverage needed, with a uniform probability distribution for all ports, it seems.
random-port() {
while [[ not != found ]]; do
# 2000..33500
port=$((RANDOM + 2000))
while [[ $port -gt 33500 ]]; do
port=$((RANDOM + 2000))
done
# 2000..65001
[[ $((RANDOM % 2)) = 0 ]] && port=$((port + 31501))
# 2000..65000
[[ $port = 65001 ]] && continue
echo $port
break
done
}
Testing
i=0
while true; do
i=$((i + 1))
printf "\rIteration $i..."
printf "%05d\n" $(random-port) >> ports.txt
done
# Then later we check the distribution
sort ports.txt | uniq -c | sort -r
You can do this
cat /dev/urandom|od -N2 -An -i|awk -v f=2000 -v r=65000 '{printf "%i\n", f + r * $1 / 65536}'
If you need more details see Shell Script Random Number Generator.
PORT=$(($RANDOM%63000+2001)) is close to what you want I think.
PORT=$(($RANDOM$RANDOM$RANDOM%63000+2001)) gets around the size limitation that troubles you. Since bash makes no distinctions between a number variable and a string variable, this works perfectly well. The "number" $RANDOM can be concatenated like a string, and then used as a number in a calculation. Amazing!
Or on OS-X the following works for me:
$ gsort --random-sort
Generate random numbers in the range [$floor,$ceil), no dependence:
$(((RANDOM % $(($ceil- $floor))) + $floor))
Generate 100 numbers between 2000 to 65000:
for i in $(seq 100); do echo $(((RANDOM % $((65000 - 2000))) + 2000));done
You can get the random number through urandom
head -200 /dev/urandom | cksum
Output:
3310670062 52870
To retrieve the one part of the above number.
head -200 /dev/urandom | cksum | cut -f1 -d " "
Then the output is
3310670062
To meet your requirement,
head -200 /dev/urandom |cksum | cut -f1 -d " " | awk '{print $1%63000+2001}'
This is how I usually generate random numbers. Then I use "NUM_1" as the variable for the port number I use. Here is a short example script.
#!/bin/bash
clear
echo 'Choose how many digits you want for port# (1-5)'
read PORT
NUM_1="$(tr -dc '0-9' </dev/urandom | head -c $PORT)"
echo "$NUM_1"
if [ "$PORT" -gt "5" ]
then
clear
echo -e "\x1b[31m Choose a number between 1 and 5! \x1b[0m"
sleep 3
clear
exit 0
fi
This works for me:
export CUDA_VISIBLE_DEVICES=$((( RANDOM % 8 )))
you can add 1 if you want it to start from 1 instead of 0.
Generating 50 numbers in Bash from a range 100000000000-999999999999 and saving them into a file filename.csv
shuf -i 100000000000-999999999999 -n 50 -o filename.csv
If you need a range bigger than 15 bit, dont use the slow unsecure and outdated 15 bit RANDOM, use the fast and secure 32 bit SRANDOM.
SRANDOM are available since about 2021 bash 5.1 roll out.
"one interesting addition to note with Bash 5.1 is the new SRANDOM variable. The SRANDOM variable provides random data from the system's entropy engine and cannot be reseeded. In particular, the SRANDOM variable provides a 32-bit random number that relies upon getrandom/getentropy -- with fall-backs to /dev/urandom or arc4random or even another fallback after that if necessary."
Source: https://www.phoronix.com/news/GNU-Bash-5.1
See what are the different of RANDOM and SRANDOM in bash:
Difference between RANDOM and SRANDOM in Bash
Feel free to improve this answer.
My.sh measures execution time in seconds:
/usr/bin/time -f '%e' long-command
I need to compare that time, that should be stored in $exptime, to some number, like:
if (( $exptime < 80 )) ; then
echo "OK" ; exit 0
elif (( $exptime > 80 )) ; then
echo "NOK" ; exit 1
else echo "error"
fi
I was trying to assign the output of a command to a variable using command substitution exptime=$(), and I get the result on console, but it doesn't work to compare that number in this script (0/1 of the script is used further). But, instead of just using the number in IF, script is called 2 more times.
There are many questions on How to set a variable to the output of a command in Bash? but I couldn't figure out the solution.
I resolved this by avoiding time and using just long-command followed by exptime=$SECONDS.
Script is iterative and SECONDS starts from 0 each time.
Not sure why time works with simple case of exptime=$( { time sleep 3 ; } 2>&1 ), where it's used as number, and not with my long-command, where echo shows it's a correct number but later in IF it repeats command with division by 0 (error token is....
I need a bash script which can generate 4500 numbers in sequence and feed it as an input to an executable and repeat the process by creating next 4500 numbers and again feed to the same executable.
The script should exit once more than 90000 numbers are generated.
Right now I am using:
i=1
while [ "$i" -le 90000 ]; do
C:/Python27/Scripts/bu.exe "$i"
i=$(($i+1))
done
which inputs one number at a time and is a time consuming process.
Any help will be gratefully appreciated.
Thanking you and regards.
$ for (( i=1; i<12; i+=3 )); do printf '####\n'; seq "$i" "$(( i+2 ))"; done
####
1
2
3
####
4
5
6
####
7
8
9
####
10
11
12
Replace the numbers above with your real values and depending on what it is you're really trying to do either pipe the seq output to your command:
$ for (( i=1; i<12; i+=3 )); do seq "$i" "$(( i+2 ))" | my_command; done
or call your command with the seq output as it's arguments:
$ for (( i=1; i<12; i+=3 )); do my_command $(seq "$i" "$(( i+2 ))"); done
I would rather advise you to execute it parallel like below. My below modification will execute 10 processes in parallel and will reduce your execution time. Please keep in mind parallel execution also depends on number of processors and memory in your system.
i=1
j=0
while [ "$i" -le 90000 ]; do
C:/Python27/Scripts/bu.exe "$i" &
# Now you are executing parallel
j=$(($j+1))
#if 10 parallel process has been created just wait to complete them
# and re-start the parallel process again
if [ $j -ge 10 ]; then
wait
j=0
fi
i=$(($i+1))
done
I read somewhere that running $(seq) is expensive (in the context of that discussion, not related to this question) so I decided to test running the seq while the bu.exe is running in the background:
for (( i=1 ; $((j=i+4499))<=90000 ; i+=4500)) # 1...4500, 4501...9000 etc.
do
parms=$(seq -s " " $i $j) # set the parameters to var
wait $pid # wait for the previous bu.exe
bu.exe "$parms" & # execute bu.exe in the bg
pid=$! # store pid
done
Sorry if the title doesn't make it clear.
I have a file to be read every 15 mins and find a particular pattern in it (e.g. timeout). File does not have any fixed update frequency.
Outcome expected:-
1. if pattern is found 3 times in 15 mins, run command1.
2. if pattern is found 5 times in 15 mins, run command2.
File to be read from last read position for each check.
Thanks,
GRV
One way to do this is with a cron job. It is more complicated than other solutions but it is very reliable (even if your "check script" crashes, it will be called up again by cron after the period elapses). The cron could be:
*/15 * * * * env DISPLAY=:0 /folder/checkscript >/dev/null 2>&1
The env DISPLAY=:0 might not be needed in your case, or it might be needed, depending on your script (note: you might need to adapt this to your case, run echo $DISPLAY to find out your variable on the case).
Your "checkscript" could be:
#!/bin/bash
if [ -f /tmp/checkVarCount ]; then oldCheckVar="$(cat /tmp/checkVarCount)"; else oldCheckVar="0"; fi
checkVar="$(grep -o "pattern" /folder/file | wc -l)"
echo "$checkVar" > /tmp/checkVarCount
checkVar="$(($checkVar-$oldCheckVar))"
if [ "$checkVar" -eq 3 ]; then command1; fi
if [ "$checkVar" -eq 5 ]; then command2; fi
exit
It is not included on your question, but if you meant to run the commands as well if the pattern is found 4 times or 7 times, then you could change the relevant parts above to:
if [ "$checkVar" -ge 3 ] && [ "$checkVar" -lt 5 ]; then command1; fi
if [ "$checkVar" -ge 5 ]; then command2; fi
I need to generate a random port number between 2000-65000 from a shell script. The problem is $RANDOM is a 15-bit number, so I'm stuck!
PORT=$(($RANDOM%63000+2001)) would work nicely if it wasn't for the size limitation.
Does anyone have an example of how I can do this, maybe by extracting something from /dev/urandom and getting it within a range?
shuf -i 2000-65000 -n 1
Enjoy!
Edit: The range is inclusive.
On Mac OS X and FreeBSD you may also use jot:
jot -r 1 2000 65000
According to the bash man page, $RANDOM is distributed between 0 and 32767; that is, it is an unsigned 15-bit value. Assuming $RANDOM is uniformly distributed, you can create a uniformly-distributed unsigned 30-bit integer as follows:
$(((RANDOM<<15)|RANDOM))
Since your range is not a power of 2, a simple modulo operation will only almost give you a uniform distribution, but with a 30-bit input range and a less-than-16-bit output range, as you have in your case, this should really be close enough:
PORT=$(( ((RANDOM<<15)|RANDOM) % 63001 + 2000 ))
and here's one with Python
randport=$(python -S -c "import random; print random.randrange(2000,63000)")
and one with awk
awk 'BEGIN{srand();print int(rand()*(63000-2000))+2000 }'
The simplest general way that comes to mind is a perl one-liner:
perl -e 'print int(rand(65000-2000)) + 2000'
You could always just use two numbers:
PORT=$(($RANDOM + ($RANDOM % 2) * 32768))
You still have to clip to your range. It's not a general n-bit random number method, but it'll work for your case, and it's all inside bash.
If you want to be really cute and read from /dev/urandom, you could do this:
od -A n -N 2 -t u2 /dev/urandom
That'll read two bytes and print them as an unsigned int; you still have to do your clipping.
If you're not a bash expert and were looking to get this into a variable in a Linux-based bash script, try this:
VAR=$(shuf -i 200-700 -n 1)
That gets you the range of 200 to 700 into $VAR, inclusive.
Here's another one. I thought it would work on just about anything, but sort's random option isn't available on my centos box at work.
seq 2000 65000 | sort -R | head -n 1
Same with ruby:
echo $(ruby -e 'puts rand(20..65)') #=> 65 (inclusive ending)
echo $(ruby -e 'puts rand(20...65)') #=> 37 (exclusive ending)
Bash documentation says that every time $RANDOM is referenced, a random number between 0 and 32767 is returned. If we sum two consecutive references, we get values from 0 to 65534, which covers the desired range of 63001 possibilities for a random number between 2000 and 65000.
To adjust it to the exact range, we use the sum modulo 63001, which will give us a value from 0 to 63000. This in turn just needs an increment by 2000 to provide the desired random number, between 2000 and 65000. This can be summarized as follows:
port=$((((RANDOM + RANDOM) % 63001) + 2000))
Testing
# Generate random numbers and print the lowest and greatest found
test-random-max-min() {
max=2000
min=65000
for i in {1..10000}; do
port=$((((RANDOM + RANDOM) % 63001) + 2000))
echo -en "\r$port"
[[ "$port" -gt "$max" ]] && max="$port"
[[ "$port" -lt "$min" ]] && min="$port"
done
echo -e "\rMax: $max, min: $min"
}
# Sample output
# Max: 64990, min: 2002
# Max: 65000, min: 2004
# Max: 64970, min: 2000
Correctness of the calculation
Here is a full, brute-force test for the correctness of the calculation. This program just tries to generate all 63001 different possibilities randomly, using the calculation under test. The --jobs parameter should make it run faster, but it's not deterministic (total of possibilities generated may be lower than 63001).
test-all() {
start=$(date +%s)
find_start=$(date +%s)
total=0; ports=(); i=0
rm -f ports/ports.* ports.*
mkdir -p ports
while [[ "$total" -lt "$2" && "$all_found" != "yes" ]]; do
port=$((((RANDOM + RANDOM) % 63001) + 2000)); i=$((i+1))
if [[ -z "${ports[port]}" ]]; then
ports["$port"]="$port"
total=$((total + 1))
if [[ $((total % 1000)) == 0 ]]; then
echo -en "Elapsed time: $(($(date +%s) - find_start))s \t"
echo -e "Found: $port \t\t Total: $total\tIteration: $i"
find_start=$(date +%s)
fi
fi
done
all_found="yes"
echo "Job $1 finished after $i iterations in $(($(date +%s) - start))s."
out="ports.$1.txt"
[[ "$1" != "0" ]] && out="ports/$out"
echo "${ports[#]}" > "$out"
}
say-total() {
generated_ports=$(cat "$#" | tr ' ' '\n' | \sed -E s/'^([0-9]{4})$'/'0\1'/)
echo "Total generated: $(echo "$generated_ports" | sort | uniq | wc -l)."
}
total-single() { say-total "ports.0.txt"; }
total-jobs() { say-total "ports/"*; }
all_found="no"
[[ "$1" != "--jobs" ]] && test-all 0 63001 && total-single && exit
for i in {1..1000}; do test-all "$i" 40000 & sleep 1; done && wait && total-jobs
For determining how many iterations are needed to get a given probability p/q of all 63001 possibilities having been generated, I believe we can use the expression below. For example, here is the calculation for a probability greater than 1/2, and here for greater than 9/10.
$RANDOM is a number between 0 and 32767. You want a port between 2000 and 65000. These are 63001 possible ports. If we stick to values of $RANDOM + 2000 between 2000 and 33500, we cover a range of 31501 ports. If we flip a coin and then conditionally add 31501 to the result, we can get more ports, from 33501 to 65001. Then if we just drop 65001, we get the exact coverage needed, with a uniform probability distribution for all ports, it seems.
random-port() {
while [[ not != found ]]; do
# 2000..33500
port=$((RANDOM + 2000))
while [[ $port -gt 33500 ]]; do
port=$((RANDOM + 2000))
done
# 2000..65001
[[ $((RANDOM % 2)) = 0 ]] && port=$((port + 31501))
# 2000..65000
[[ $port = 65001 ]] && continue
echo $port
break
done
}
Testing
i=0
while true; do
i=$((i + 1))
printf "\rIteration $i..."
printf "%05d\n" $(random-port) >> ports.txt
done
# Then later we check the distribution
sort ports.txt | uniq -c | sort -r
You can do this
cat /dev/urandom|od -N2 -An -i|awk -v f=2000 -v r=65000 '{printf "%i\n", f + r * $1 / 65536}'
If you need more details see Shell Script Random Number Generator.
PORT=$(($RANDOM%63000+2001)) is close to what you want I think.
PORT=$(($RANDOM$RANDOM$RANDOM%63000+2001)) gets around the size limitation that troubles you. Since bash makes no distinctions between a number variable and a string variable, this works perfectly well. The "number" $RANDOM can be concatenated like a string, and then used as a number in a calculation. Amazing!
Or on OS-X the following works for me:
$ gsort --random-sort
Generate random numbers in the range [$floor,$ceil), no dependence:
$(((RANDOM % $(($ceil- $floor))) + $floor))
Generate 100 numbers between 2000 to 65000:
for i in $(seq 100); do echo $(((RANDOM % $((65000 - 2000))) + 2000));done
You can get the random number through urandom
head -200 /dev/urandom | cksum
Output:
3310670062 52870
To retrieve the one part of the above number.
head -200 /dev/urandom | cksum | cut -f1 -d " "
Then the output is
3310670062
To meet your requirement,
head -200 /dev/urandom |cksum | cut -f1 -d " " | awk '{print $1%63000+2001}'
This is how I usually generate random numbers. Then I use "NUM_1" as the variable for the port number I use. Here is a short example script.
#!/bin/bash
clear
echo 'Choose how many digits you want for port# (1-5)'
read PORT
NUM_1="$(tr -dc '0-9' </dev/urandom | head -c $PORT)"
echo "$NUM_1"
if [ "$PORT" -gt "5" ]
then
clear
echo -e "\x1b[31m Choose a number between 1 and 5! \x1b[0m"
sleep 3
clear
exit 0
fi
This works for me:
export CUDA_VISIBLE_DEVICES=$((( RANDOM % 8 )))
you can add 1 if you want it to start from 1 instead of 0.
Generating 50 numbers in Bash from a range 100000000000-999999999999 and saving them into a file filename.csv
shuf -i 100000000000-999999999999 -n 50 -o filename.csv
If you need a range bigger than 15 bit, dont use the slow unsecure and outdated 15 bit RANDOM, use the fast and secure 32 bit SRANDOM.
SRANDOM are available since about 2021 bash 5.1 roll out.
"one interesting addition to note with Bash 5.1 is the new SRANDOM variable. The SRANDOM variable provides random data from the system's entropy engine and cannot be reseeded. In particular, the SRANDOM variable provides a 32-bit random number that relies upon getrandom/getentropy -- with fall-backs to /dev/urandom or arc4random or even another fallback after that if necessary."
Source: https://www.phoronix.com/news/GNU-Bash-5.1
See what are the different of RANDOM and SRANDOM in bash:
Difference between RANDOM and SRANDOM in Bash
Feel free to improve this answer.