var+= adds string instead of sum - shell

I have to make a script that makes the sum of all processes that have PIDS greater than 20. It kind of works but something is going wrong. I know it's something simple, base thing, but I can't get the logic.
For example I have 2 processes, "1504" and "1405". My result of sum is 15041405. How do I have to re-write sum+=${proc[$i]} to make an actual sum, not string attach?
proc=( $(ps | cut -d ' ' -f1) )
nr=$(ps | cut -d ' ' -f1 | wc -l)
sum=0
for (( i=0 ; i<=nr ; i++)); do
[[ ${proc[$i]} -gt 20 ]] && sum+=${proc[$i]}
done
echo $sum

Use a math context. In POSIX sh:
sum=$(( sum + val ))
...or, also valid POSIX:
: "$(( sum += val ))"
...or, in bash:
(( sum += val ))
You can also use much easier-to-read comparison operations in a math context, rather than using -gt inside of a non-math test context. In bash:
(( ${proc[$i]} >= 20 )) && (( sum += ${proc[$i]} ))
...or in POSIX shell (which doesn't support arrays, and so cannot exactly reproduce your sample code):
: "$(( sum += ( val < 20 ) ? 0 : val ))"
If you were trying to do the (more sensible) operation of counting PIDs, I'd consider an implementation more like the following (bash-only and Linux-only, but considerably more efficient):
count=0
for pid_file in /proc/[0-9]*; do
pid=${pid_file##*/}
(( pid > 20 )) && (( count++ ))
done
printf '%s\n' "$count"
...or, to put more of the effort on the glob engine:
# avoid inflating the result with non-matching globs
shopt -s nullglob
# define a function to avoid overwriting the global "$#" array
count_procs() {
set -- /proc/[3-9][0-9] /proc/[0-9][0-9][0-9]*
echo "$#"
}
# ...used as follows:
count_procs

Related

How do I create large CSVs in seconds?

I am trying to create 1000s of large CSVs rapidly. This function generates the CSVs:
function csvGenerator () {
for ((i=1; i<=$NUMCSVS; i++)); do
CSVNAME=$DIRNAME"-"$CSVPREFIX$i$CSVEXT
HEADERARRAY=()
if [[ ! -e $CSVNAME ]]; then #Only create csv file if it not exist
touch $CSVNAME
echo "file: "$CSVNAME "created at $(date)" >> ../status.txt
fi
for ((j=1; j<=$NUMCOLS; j++)); do
if (( j < $NUMCOLS )) ; then
HEADERNAME=$DIRNAME"-csv-"$i"-header-"$j", "
elif (( j == $NUMCOLS )) ; then
HEADERNAME=$DIRNAME"-csv-"$i"-header-"$j
fi
HEADERARRAY+=$HEADERNAME
done
echo $HEADERARRAY > $CSVNAME
for ((k=1; k<=$NUMROWS; k++)); do
ROWARRAY=()
for ((l=1; l<=$NUMCOLS; l++)); do
if (( l < $NUMCOLS )) ; then
ROWVALUE=$DIRNAME"-csv-"$i"-r"$k"c"$l", "
elif (( l == $NUMCOLS )) ; then
ROWVALUE=$DIRNAME"-csv-"$i"-r"$k"c"$l
fi
ROWARRAY+=$ROWVALUE
done
echo $ROWARRAY >> $CSVNAME
done
done
}
The script takes ~3 mins to generate a CSV with 100k rows and 70 cols. What do I need to do to generate these CSVs at the rate of 1 CSV/~10 seconds?
Let me start by saying that bash and "performant" don't usually go together in the same sentence. As other commentators suggested, awk may be a good choice that's adjacent in some senses.
I haven't yet had a chance to run your code, but it opens and closes the output file once per row — in this example, 100,000 times. Each time it must seek to the end of the file so that it can append the latest row.
Try pulling the actual generation (everything after for ((j=1; j<=$NUMCOLS; j++)); do) into a new function, like generateCsvContents. In that new function, don't reference $CSVNAME, and remove the redirections on the echo statements. Then, in the original function, call the new function and redirect its output to the filename. Roughly:
function csvGenerator () {
for ((i=1; i<=NUMCSVS; i++)); do
CSVNAME=$DIRNAME"-"$CSVPREFIX$i$CSVEXT
if [[ ! -e $CSVNAME ]]; then #Only create csv file if it not exist
echo "file: $CSVNAME created at $(date)" >> ../status.txt
fi
# This will create $CSVNAME if it doesn't yet exist
generateCsvContents > "$CSVNAME"
done
}
function generateCsvContents() {
HEADERARRAY=()
for ((j=1; j<=NUMCOLS; j++)); do
if (( j < NUMCOLS )) ; then
HEADERNAME=$DIRNAME"-csv-"$i"-header-"$j", "
elif (( j == NUMCOLS )) ; then
HEADERNAME=$DIRNAME"-csv-"$i"-header-"$j
fi
HEADERARRAY+=$HEADERNAME
done
echo $HEADERARRAY
for ((k=1; k<=NUMROWS; k++)); do
ROWARRAY=()
for ((l=1; l<=NUMCOLS; l++)); do
if (( l < NUMCOLS )) ; then
ROWVALUE=$DIRNAME"-csv-"$i"-r"$k"c"$l", "
elif (( l == NUMCOLS )) ; then
ROWVALUE=$DIRNAME"-csv-"$i"-r"$k"c"$l
fi
ROWARRAY+=$ROWVALUE
done
echo "$ROWARRAY"
done
}
"Not this way" is I think the answer.
There are a few problems here.
You're not using your arrays as arrays. When you treat them like strings, you affect only the first element in the array, which is misleading.
The way you're using >> causes the output file to be opened and closed once for every line. That's potentially wasteful.
You're not quoting your variables. In fact, you're quoting the stuff that doesn't need quoting, and not quoting the stuff that does.
Upper case variable names are not recommended, due to the risk of collision with system variables. ref
Bash isn't good at this. Really.
A cleaned up version of your function might look like this:
csvGenerator2() {
for (( i=1; i<=NUMCSVS; i++ )); do
CSVNAME="$DIRNAME-$CSVPREFIX$i$CSVEXT"
# Only create csv file if it not exist
[[ -e "$CSVNAME" ]] && continue
touch "$CSVNAME"
date "+[%F %T] created: $CSVNAME" | tee -a status.txt >&2
HEADER=""
for (( j=1; j<=NUMCOLS; j++ )); do
printf -v HEADER '%s, %s-csv-%s-header-%s' "$HEADER" "$DIRNAME" "$i" "$j"
done
echo "${HEADER#, }" > "$CSVNAME"
for (( k=1; k<=NUMROWS; k++ )); do
ROW=""
for (( l=1; l<=NUMCOLS; l++ )); do
printf -v ROW '%s, %s-csv-%s-r%sc%s' "$ROW" "$DIRNAME" "$i" "$k" "$l"
done
echo "${ROW#, }"
done >> "$CSVNAME"
done
}
(Note that I haven't switched the variables to lower case because I'm lazy, but it's still a good idea.)
And if you were to make something functionally equivalent in awk:
csvGenerator3() {
awk -v NUMCSVS="$NUMCSVS" -v NUMCOLS="$NUMCOLS" -v NUMROWS="$NUMROWS" -v DIRNAME="$DIRNAME" -v CSVPREFIX="$CSVPREFIX" -v CSVEXT="$CSVEXT" '
BEGIN {
for ( i=1; i<=NUMCSVS; i++) {
out=sprintf("%s-%s%s%s", DIRNAME, CSVPREFIX, i, CSVEXT)
if (!system("test -e " CSVNAME)) continue
system("date '\''+[%F %T] created: " out "'\'' | tee -a status.txt >&2")
comma=""
for ( j=1; j<=NUMCOLS; j++ ) {
printf "%s%s-csv-%s-header-%s", comma, DIRNAME, i, j > out
comma=", "
}
printf "\n" >> out
for ( k=1; k<=NUMROWS; k++ ) {
comma=""
for ( l=1; l<=NUMCOLS; l++ ) {
printf "%s%s-csv-%s-r%sc%s", comma, DIRNAME, i, k, l >> out
comma=", "
}
printf "\n" >> out
}
}
}
'
}
Note that awk does not suffer from the same open/closer overhead mentioned earlier with bash; when a file is used for output or as a pipe, it gets opened once and is left open until it is closed.
Comparing the two really highlights the choice you need to make:
$ time bash -c '. file; NUMCSVS=1 NUMCOLS=10 NUMROWS=100000 DIRNAME=2 CSVPREFIX=x CSVEXT=.csv csvGenerator2'
[2019-03-29 23:57:26] created: 2-x1.csv
real 0m30.260s
user 0m28.012s
sys 0m1.395s
$ time bash -c '. file; NUMCSVS=1 NUMCOLS=10 NUMROWS=100000 DIRNAME=3 CSVPREFIX=x CSVEXT=.csv csvGenerator3'
[2019-03-29 23:58:23] created: 3-x1.csv
real 0m4.994s
user 0m3.297s
sys 0m1.639s
Note that even my optimized bash version is only a little faster than your original code.
Refactoring your two inner for-loops to loops like this will save time:
for ((j=1; j<$NUMCOLS; ++j)); do
HEADERARRAY+=$DIRNAME"-csv-"$i"-header-"$j", "
done
HEADERARRAY+=$DIRNAME"-csv-"$i"-header-"$NUMCOLS

trouble debugging a .bash file that separates and echoes either student or student score

Like the title implies, I'm currently having trouble debugging a .bash file. As you can tell, I'm still very new to bash and unix in general. Inside The inputfile that hw3.bash executes on contains the following strings in two columns:
jack 80
mary 95
michael 60
jeffrey 90
The file will print out the names only, then scores only, then highest score, then lowest, then rank, etc.
Both lines 72 and 150 are the hiccups in the program that i cannot seem to debug. Is this a indentation issue or simply a grammatical issue.
line 72: unexpected EOF while looking for matching `''
line 150: syntax error: unexpected end of file
#1.The first positional argument ($1) provides the file name. Assign it to a
# variable named inputfile.
inputfile=$1
#2.Test if the variable inputfile refers to a file. If not, print out a proper
# message and exit with an exit code of 1.
if [[ -e inputfile ]]; then
echo "inputfile is not a file'
exit 1
fi
#3.Suppose each line of the input file is made up of the first name of a
#student followed by a score (the first name and score are separate by a
# space). Put the first names in the input file into an array called names.
record=( $(cat $inputfile | awk 'END{print NR}' ))
names=( $(cat $inputfile | awk'{print $1}'))
echo ${names[*]}
#4.Using a similar approach, put the corresponding scores into an
# array called scores.
scores=( $(cat $inputfile | awk'{print $2}'))
echo ${scores[*]}
#5.Print names and corresponding scores (i.e. output should look the same as
# the input file). You must use a loop to do this.
names=( $(cat $inputfile | awk'{print $1}'))
echo${names[*]}
scores=( $(cat $inputfile | awk'{print $2}'))
echo${names[*]}
for((i=0;i<${names[*] && $i<scores[*];i++))
do
echo{$names[i]} {$scores[*]}
done
#6.Find the highest scorer, and print the name and the score. You may assume
#that there is only one highest score.
maxVal=1
maxIndex=1
for(( i=0; i<$#names[*]} && i<${#scores[*]}; i++ ))
do
if [[ (-n ${scores[$i]}) ]]
then
if(( ${scores[$i]} > $maxValue ))
then
maxVal=${scores[$i]}
maxIndex=$i
fi
fi
done
echo "Highest Scorer:" ${names[$maxIndex]} " " $maxVal
#7.Find the lowest scorer, and print the name and the score. You may assume
#that there is only one lowest score.
minVal=10000
maxIndex=1
for(( i=0; i<${#names[*]} && i<${#scores[*]}; i++ ))
do
if [[ (-n ${scores[$i]}) ]]
then
if(( $minVal > ${scores[i]} ))
then
minVal=${scores[$i]}
minIndex=$i;
fi
done
echo "Lower Scorer:' ${names[$minIndex]} " " $minVal
#8.Calculate the average of all scores, and print it.
avg=0
total=0;
for(( i=0;i<${#names[*]} && i<${$#scores[*]};i++ ))
do
if [[ (-n ${scores[$i]} ]]
then
total=$(( $total + ${scores[$i]} ))
fi
#9.Sort the arrays in terms of scores (in descending order). The final
#position of each name in the array names must match with the position of the
# corresponding score in the array scores.
m=${names[*]}
n=${scores[*]}
for(( i=0; i<$n && i<$m; i++ ))
do
maxValue=1
maxIndex=0
for(( i=0; j<$n && j<$m; j++))
do
if [[ !(-z ${scores[$j]} ) ]]
then
if (( $maxVal < ${scores[$j]} ))
then
maxVal=${scores[$j]}
maxIndex=$j
fi
fi
done
a1[${#a1[*]}]=$maxVal
a2[${#a2[*]}]=${names[$maxIndex]}
unset scores[$maxIndex]
done
for(( i=0; j<${#a2[*]} && i < ${#a1[*]; i++ ))
do
echo ${a2[i]} ${a1[i]}
done
#10.Print sorted names and scores. Put the rank of each student before the
m=${#names[*]}
n=${#scores[*]}
for (( i=0; i<$n && i<$m; i++ ))
do
maxVal=1
maxIndex=0
for (( j=0: j<$n && j<$m; j++ ))
do
if [[ !_-z ${scores[$j]}) ]]
then
if (( $maxVal < ${scores[$j]}
maxIndex=$j
fi
fi
done
a1[${#a1[*]}]=$maxVal
a2[${#a2[*]}=${names[$maxIndex]}
unset scores[$maxIndex]
done
k=1
while [[ $k -lt ${#a2[*]} ]]
do
for(( i=0;i < ${#a2[*]} && i < ${#a1[*]};i++ ))
do
echo "Ranking" $k "is:" ${a2[i]}
k=$(($k+1))
done
done
Like you can see in your line, just before the then one, you mixed single and double quotes:
echo "inputfile is not a file'
Just replace the final simple quote, by a double one.

(standard_in) 1: syntax error, comparing floating point no using bc in bash [duplicate]

I have this script that should make sure that the users current PHP version is between a certain range, though it SHOULD work, there is a bug somewhere that makes it think that the version is out of range, could someone take a look and tell me what I can do to fix it?
function version { echo "$#" | gawk -F. '{ printf("%d.%d.%d\n", $1,$2,$3); }'; }
phpver=`php -v |grep -Eow '^PHP [^ ]+' |gawk '{ print $2 }'`
if [ $(version $phpver) > $(version 5.2.13) ] || [ $(version $phpver) < $(version 5.2.13) ]; then
echo "PHP Version $phpver must be between 5.2.13 - 5.3.15"
exit
fi
Here's how to compare versions.
using sort -V:
function version_gt() { test "$(printf '%s\n' "$#" | sort -V | head -n 1)" != "$1"; }
example usage:
first_version=5.100.2
second_version=5.1.2
if version_gt $first_version $second_version; then
echo "$first_version is greater than $second_version !"
fi
pro:
solid way to compare fancy version strings:
support any length of sub-parts (ie: 1.3alpha.2.dev2 > 1.1 ?)
support alpha-betical sort (ie: 1.alpha < 1.beta2)
support big size version (ie: 1.10003939209329320932 > 1.2039209378273789273 ?)
can easily be modified to support n arguments. (leaved as an exercise ;) )
usually very usefull with 3 arguments: (ie: 1.2 < my_version < 2.7 )
cons:
uses a lot of various calls to different programs. So it's not that efficient.
uses a pretty recent version of sort and it might not be available on your
system. (check with man sort)
without sort -V:
## each separate version number must be less than 3 digit wide !
function version { echo "$#" | gawk -F. '{ printf("%03d%03d%03d\n", $1,$2,$3); }'; }
example usage:
first_version=5.100.2
second_version=5.1.2
if [ "$(version "$first_version")" -gt "$(version "$second_version")" ]; then
echo "$first_version is greater than $second_version !"
fi
pro:
quicker solution as it only calls 1 subprocess
much more compatible solution.
cons:
quite specific, version string must:
have version with 1, 2 or 3 parts only. (excludes '2.1.3.1')
each parts must be numerical only (excludes '3.1a')
each part can't be greater than 999 (excludes '1.20140417')
Comments about your script:
I can't see how it could work:
as stated in a comment > and < are very special shell character and you should replace them by -gt and -lt
even if you replaced the characters, you can't compare version numbers as if they where integers or float. For instance, on my system, php version is 5.5.9-1ubuntu4.
But your function version() is quite cleverly written already and may help you by circumventing the classical issue that sorting alphabetically numbers won't sort numbers numerically ( alphabetically 1 < 11 < 2, which is wrong numerically). But be carefull: arbitrarily large numbers aren't supported by bash (try to keep under 32bits if you aim at compatibility with 32bits systems, so that would be 9 digit long numbers). So I've modified your code (in the second method NOT using sort -V) to force only 3 digits for each part of the version string.
EDIT: applied #phk amelioration, as it is noticeably cleverer and remove a subprocess call in the first version using sort. Thanks.
There is possibility for deb-distributions:
dpkg --compare-versions <version> <relation> <version>
For example:
dpkg --compare-versions "0.0.4" "gt" "0.0.3"
if [ $? -eq "0" ]; then echo "YES"; else echo "NO"; fi
It is doing a lexical comparison. Use one of these:
if [ $(version $phpver) -gt $(version 5.2.13) ] || [ $(version $phpver) -lt $(version 5.2.13) ]; then
if [[ $(version $phpver) > $(version 5.2.13) ]] || [[ $(version $phpver) < $(version 5.2.13) ]]; then
if (( $(version $phpver) > $(version 5.2.13) )) || (( $(version $phpver) < $(version 5.2.13) )); then
Or do it all in awk or some other tool. It is screaming for some optimisation. It also seems you're not producing numbers either, so you have a pretty odd design. Usually the version substrings are multiplied by 1000 and then all summed up to get a single comparable scalar.
Here's another solution that:
does not run any external command apart from tr
has no restriction on number of parts in version string
can compare version strings with different number of parts
Note that it's Bash code using array variables.
compare_versions()
{
local v1=( $(echo "$1" | tr '.' ' ') )
local v2=( $(echo "$2" | tr '.' ' ') )
local len="$(max "${#v1[*]}" "${#v2[*]}")"
for ((i=0; i<len; i++))
do
[ "${v1[i]:-0}" -gt "${v2[i]:-0}" ] && return 1
[ "${v1[i]:-0}" -lt "${v2[i]:-0}" ] && return 2
done
return 0
}
The function returns:
0 if versions are equal (btw: 1.2 == 1.2.0)
1 if the 1st version is bigger / newer
2 if the 2nd version is bigger / newer
However #1 -- it requires one additional function (but function min is quite usable to have anyway):
min()
{
local m="$1"
for n in "$#"
do
[ "$n" -lt "$m" ] && m="$n"
done
echo "$m"
}
However #2 -- it cannot compare version strings with alpha-numeric parts (though that would not be difficult to add, actually).
A much safer option for testing the PHP CLI version is to use PHP's own version_compare function.
#!/bin/bash
MIN_VERSION="7.0.0"
MAX_VERSION="7.1.4"
PHP_VERSION=`php -r 'echo PHP_VERSION;'`
function version_compare() {
COMPARE_OP=$1;
TEST_VERSION=$2;
RESULT=$(php -r 'echo version_compare(PHP_VERSION, "'${TEST_VERSION}'", "'${COMPARE_OP}'") ? "TRUE" : "";')
test -n "${RESULT}";
}
if ( version_compare "<" "${MIN_VERSION}" || version_compare ">" "${MAX_VERSION}" ); then
echo "PHP Version ${PHP_VERSION} must be between ${MIN_VERSION} - ${MAX_VERSION}";
exit 1;
fi
echo "PHP Version ${PHP_VERSION} is good!";
The following solution should more accurately addresses your exact need. It can be used to test whether the CURRENT version string falls between MIN and MAX. I am assuming that MIN and MAX are acceptable version numbers (i.e. MIN <= CURRENT <= MAX rather than MIN < CURRENT < MAX).
# Usage: version MIN CURRENT MAX
version(){
local h t v
[[ $2 = "$1" || $2 = "$3" ]] && return 0
v=$(printf '%s\n' "$#" | sort -V)
h=$(head -n1 <<<"$v")
t=$(tail -n1 <<<"$v")
[[ $2 != "$h" && $2 != "$t" ]]
}
For example...
if ! version 5.2.13 "$phpver" 5.3.15; then
echo "PHP Version $phpver must be between 5.2.13 - 5.3.15"
exit
fi
If you're on Bash 3 with an older version of sort (lookin at you macOS...), then I created the following helper script you can source in (can also be ran as a command):
https://github.com/unicorn-fail/version_compare
I wrote this inelegant function a while back for a similar problem. vers_limit will return 0 if arg1 is less than or equal to arg2, non-0 otherwise:
vers_limit()
{
VERNEW=$1
VERLMT=$2
CHKNEW=$VERNEW
CHKLMT=$VERLMT
PASSED=
while :
do
PARTNEW=${CHKNEW%%.*}
PARTLMT=${CHKLMT%%.*}
if [[ $PARTNEW -lt $PARTLMT ]]
then
PASSED=GOOD
break
elif [[ $PARTNEW -gt $PARTLMT ]]
then
PASSED=
break
else
NXTNEW=${CHKNEW#*.}
if [[ $NXTNEW == $CHKNEW ]]
then
if [[ $NXTNEW == $CHKLMT ]]
then
PASSED=GOOD
break
else
NXTNEW=0
fi
fi
NXTLMT=${CHKLMT#*.}
if [[ $NXTLMT == $CHKLMT ]]
then
NXTLMT=0
fi
fi
CHKNEW=$NXTNEW
CHKLMT=$NXTLMT
done
test "$PASSED"
}
This is not as compact as some of the other solutions here, nor does it provide 3-way status (i.e., less, equal, greater), though I believe one can order the arguments, interpret the pass/fail status, and/or call twice to accomplish any desired result. That said, vers_limit does have certain virtues:
No calls to external utilities such as sort, awk, gawk, tr, etc.
Handles numeric versions of arbitrary size (up to the shell's
limit for integer calculations)
Handles an arbitrary number of "dotted" parts.
v_min="5.2.15"
v_max="5.3.15"
v_php="$(php -v | head -1 | awk '{print $2}')"
[ ! "$v_php" = "$(echo -e "$v_php\n$v_min\n$v_max" | sort -V | head -2 | tail -1)" ] && {
echo "PHP Version $v_php must be between $v_min - $v_max"
exit
}
This puts v_min, v_max and v_php in version order and tests if v_php is not in the middle.
PURE BASH
I found my way to this page because I had the same problem. The other answers did not satisfy me, so I wrote this function.
So long as the 2 versions have the same number of periods in them this will compare the versions correctly.
It does a c style for loop, setting $i incrementally from 0 to # of numbers in the version string.
for each #:
if new - old is neg we know the first version is newer.
If new - old is pos we know the first version is older.
If new - old is 0 then it is the same and we need to continue checking.
We run false after to set exit status of the function for the case where $1 == $2 the versions are totally identical.
newver=1.10.1
installedver=1.9.25
#installedver=1.11.25
#installedver=1.10.1
checkupdate(){
# $1 = new version
# $2 = installed version
IFS='.' read -r -a nver <<< "$1"
IFS='.' read -r -a iver <<< "$2"
for ((i = 0 ; i < "${#nver[#]}" ; i++)) ;do
case "$((${nver[i]}-${iver[i]}))" in
-*) return 1 ;;
0) ;;
*) return 0 ;;
esac
false
done
}
checkupdate "$newver" "$installedver" && echo yes || echo no
Another method for SH
After I tried to implement my above function on Android I realized that I would not always have bash, so the above function did not work for me. Here is the version I wrote using awk to get around needing bash:
checkupdate(){
# $1 = new version
# $2 = installed version
i=1
#we start at 1 and go until number of . so we can use our counter as awk position
places=$(awk -F. '{print NF+1}' <<< "$1")
while (( "$i" < "$places" )) ;do
npos=$(awk -v pos=$i -F. '{print $pos}' <<< "$1")
ipos=$(awk -v pos=$i -F. '{print $pos}' <<< "$2")
case "$(( $npos - $ipos ))" in
-*) return 1 ;;
0) ;;
*) return 0 ;;
esac
i=$((i+1))
false
done
}

Comparison between array items

I've written a script to calculate the bandwidth usage of an OpenVZ container over time and suspend it if it uses too much too quickly. Here is the script so far:
#!/bin/bash
# Thresholds are in bytes per second
LOGDIR="/var/log/outbound_ddos"
THRESHOLD1=65536
THRESHOLD2=117964
while [ 1 ]
do
for veid in $(/usr/sbin/vzlist -o veid -H)
do
# Create the log file if it doesn't already exist
if ! test -e $LOGDIR/$veid.log; then
touch $LOGDIR/$veid.log
fi
# Parse out the inbound/outbound traffic and assign them to the corresponding variables
eval $(/usr/sbin/vzctl exec $veid "grep venet0 /proc/net/dev" | \
awk -F: '{print $2}' | awk '{printf"CTOUT=%s\n", $9}')
# Print the output and a timestamp to a log file
echo $(date +%s) $CTOUT >> $LOGDIR/$veid.log
# Read last 10 entries into arrays
i=0
tail $LOGDIR/$veid.log | while read time byte
do
times[i]=$time
bytes[i]=$byte
let ++i
done
# Time checks & calculations for higher threshold
counter=0
for (( i=0; i<9; i++ ))
do
# If we have roughly the right timestamp
if (( times[9-i] < times[8-i] + 20 ))
then
# If the user has gone over the threshold
if (( bytes[9-i] > bytes[8-i] + THRESHOLD2 * 10 ))
then let ++counter
fi
fi
done
# Now check counter
if (( counter == 9 ))
then vzctl stop $veid
fi
# Same for lower threshold
counter=0
for (( i=0; i<3; i++ ))
do
# If we have roughly the right timestamp
if (( times[3-i] < times[2-i] + 20 ))
then
# If the user has gone over the threshold
if (( bytes[3-i] > bytes[2-i] + THRESHOLD1 * 10 ))
then let ++counter
fi
fi
done
# Now check counter
if (( counter == 2 ))
then vzctl stop $veid
fi
done
sleep 10
done
I've checked the numbers in /var/log/outbound_ddos/vm101.log and they're increasing by more than the threshold, but nothing is happening.
I added some echo statements to try and figure out where the problem is and it seems to be this comparison that's returning false:
if (( bytes[9-i] > bytes[8-i] + THRESHOLD2 * 10 ))
So then I tried the following, which printed out nothing:
echo ${bytes[9-i]}
Could anyone point me in the right direction? I think the script is nearly done, probably something very simple.
Your shell runs the while read loop in a subshell (see here for why it does not work as expected), so your array magic does not propagate outside the tail | while construct.
Read this and fix accordingly :-)

Parameter input works, but pipe doesn't

I tried to create a shell script, which sum the given numbers. If there is no given parameter, then it tries to read the pipe output, but I get an error.
#!/bin/sh
sum=0
if [ $# -eq 0 ]
then
while read data
do
sum=`expr $sum + $data`
done
else
for (( i = 1 ; i <= $#; i++ ))
do
sum=`expr $sum + ${!i}`
done
fi
echo $sum
This works: sum 10 12 13
But this one doesn't: echo 10 12 13| sum
Thanks in advance,
Here you go (assuming bash, not sh):
#!/bin/bash
sum=0
if (( $# == 0 )); then
# Read line by line
# But each line might consist of separate numbers to be added
# So read each line as an array!
while read -a data; do
# Now data is an array... but if empty, continue
(( ${#data[#]} )) || continue
# Convert this array into a string s, with elements separated by a +
printf -v s "%s+" ${data[#]}
# Append 0 to s (observe that s ended with a +)
s="${s}0"
# Add these numbers to sum
(( sum += s ))
done
else
# If elements come from argument line, do the same!
printf -v s "%s+" $#
# Append 0 to s (observe that s ended with a +)
s="${s}0"
# Add these numbers to obtain sum
(( sum = s ))
fi
echo $sum
You can invoke it thus:
$ echo 10 12 13 | ./sum
35
$ ./sum 10 12 13
35
$ # With several lines and possibly empty lines:
$ { echo 10 12 13; echo; echo 42 22; } | ./sum
99
Hope this helps!
Edit. You might also be interested in learning cool stuff about IFS. I've noticed that people tend to confuse # and * in bash. If you don't know what I'm talking about, then you should use # instead of *, also for array subscripts! In the bash manual, you'll find that when double quoted, $* (or ${array[*]}) expands to all the elements of the array separated by the value of the IFS. This can be useful in our case:
#!/bin/bash
sum=0
if (( $# == 0 )); then
# Read line by line
# But each line might consist of separate numbers to be added
# So read each line as an array!
while read -a data; do
# Now data is an array... but if empty, continue
(( ${#data[#]} )) || continue
# Setting IFS=+ (just for the sum) will yield exactly what I want!
IFS=+ sum=$(( sum + ${data[*]} ))
done
else
# If elements come from argument line, do the same!
# Setting IFS=+ (just for the sum) will yield exactly what I want!
IFS=+ sum=$(( $* ))
fi
echo $sum
Gniourf now exits from teacher mode. :-)

Resources