Why is my code not working as I want it to? - bash

I have this code:
total=0;
ps -u $(whoami) --no-headers | awk {'print $1'} | while read line;
do vrednost=$(pmap $line | tail -n1 | column -t | cut -d" " -f3 | tr "K" " ");
total=$(( vrednost + total ))
echo $total
done
echo total: $total
As you can see, my code sums usage of all my processes. When I echo my total every time in while, it is working ok, but at the end... When i want total to be a value (echo total: $total) it is still zero. but before (in while) has right value.

BASH FAQ #24: "I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?"
#!/bin/bash
while read ...
do
...
done < <(ps ...)

Okay, pick and choose. You can either do it in BASH or AWK, but don't do both. You've seen a BASH example, here's an AWK example:
ps -e -o user -o vsz | awk -v USER="$(whoami)" '
BEGIN {TOTAL = 0}
END {print "Total is " TOTAL}
{
if ($1 == USER) {
TOTAL += $2
}
}
'
Awk is like a programming language that assumes a loop (like perl -n) and processes each line in the file. Each field (normally separated by whitespace) is given a $ variable. The first is $1, the second is $2, etc.
The -v option allows me to define an awk variable (in this case USER) before I run awk.
The BEGIN line is what I want to do before I run my awk script. In this case, initialize TOTAL to zero. (NOTE: This really isn't necessary since undefined variables automatically are given a value of zero). The END line is what I want to do afterwards. In this case, print out my total.
So, if the first field ($1) is equal to my user, I'll add the second field (the vsize) to my total.
All Awk programs are surrounded by {...} and they usually have single quotes around them to prevent shell interpolation of $1, etc.

Try this
#!/bin/bash
total=0;
for line in `ps -u $(whoami) --no-headers | awk {'print $1'}`;
do
vrednost=$(pmap $line | tail -n1 | column -t | cut -d" " -f3 | tr "K" " ");
total=$(( $vrednost + $total ))
echo $total
done
echo "total: $total"

Ignacio's answer is fine, but process substitution is not portable. And there is a simpler solution. You need to echo the total in the same subshell in which it is calculated:
... | { while read line; do ...; done; echo total: $total; }

Let's cut down on the number of extra processes you need :)
declare -i total=0
for size in $( ps -u $(whoami) --no-header -o vsz ); do
total+=$size
done
echo $total
First, use various options for ps to generate the desired list of process sizes, in kilobytes. Iterate over that list using a bash for-loop, keeping a running total in a parameter declared with the 'integer' attribute for easy arithmetic. The desired sum is now in total, ready for whatever use you need. The sum includes the memory used by the ps process itself.
Using while (Dennis' suggestion) and avoiding process substitution (William's suggestion):
ps -u $(whoami) --no-header -o vsz | {
while read -r var; do
((total+=$var))
done
echo $total
}
(For real one-liner, here's a dc command that I borrowed from https://stackoverflow.com/a/453290/1126841:
ps -u $(whoami) --no-header -o vsz | dc -f - -e '[+z1<r]srz1<rp'
This sum includes the memory used by the ps and dc commands themselves.)

Related

Counting all the 5 from a specific range in Bash

I want to count how many times the digit "5" appears from the range 1 to 4321. For example, the number 5 appears 1 or the number 555, 5 would appear 3 times etc.
Here is my code so far, however, the results are 0, and they are supposed to be 1262.
#!/bin/bash
typeset -i count5=0
for n in {1..4321}; do
echo ${n}
done | \
while read -n1 digit ; do
if [ `echo "${digit}" | grep 5` ] ; then
count5=count5+1
fi
done | echo "${count5}"
P.s. I am looking to fix my code so it can print the right output. I do not want a completely different solution or a shortcut.
What about something like this
seq 4321 | tr -Cd 5 | wc -c
1262
Creates the sequence, delete everything but 5's and count the chars
The main problem here is http://mywiki.wooledge.org/BashFAQ/024. With minimal changes, your code could be refactored to
#!/bin/bash
typeset -i count5=0
for n in {1..4321}; do
echo $n # braces around ${n} provide no benefit
done | # no backslash required here; fix weird indentation
while read -n1 digit ; do
# prefer modern command substitution syntax over backticks
if [ $(echo "${digit}" | grep 5) ] ; then
count5=count5+1
fi
echo "${count5}" # variable will not persist outside subprocess
done | head -n 1 # so instead just print the last one after the loop
With some common antipatterns removed, this reduces to
#!/bin/bash
printf '%s\n' {1..4321} |
grep 5 |
wc -l
A more efficient and elegant way to do the same is simply
printf '%s\n' {1..4321} | grep -c 5
One primary issue:
each time results are sent to a pipe said pipe starts a new subshell; in bash any variables set in the subshell are 'lost' when the subshell exits; net result is even if you're correctly incrementing count5 within a subshell you'll still end up with 0 (the starting value) when you exit from the subshell
Making minimal changes to OP's current code:
while read -n1 digit ; do
if [ `echo "${digit}" | grep 5` ]; then
count5=count5+1
fi
done < <(for n in {1..4321}; do echo ${n}; done)
echo "${count5}"
NOTE: there are a couple performance related issues with this method of coding but since OP has explicitly asked to a) 'fix' the current code and b) not provide any shortcuts ... we'll leave the performance fixes for another day ...
A simpler way to get the number for a certain n would be
nx=${n//[^5]/} # Remove all non-5 characters
count5=${#nx} # Calculate the length of what is left
A simpler method in pure bash could be:
printf -v seq '%s' {1..4321} # print the sequence into the variable seq
fives=${seq//[!5]} # delete all characters but 5s
count5=${#fives} # length of the string is the count of 5s
echo $count5 # print it
Or, using standard utilities tr and wc
printf '%s' {1..4321} | tr -dc 5 | wc -c
Or using awk:
awk 'BEGIN { for(i=1;i<=4321;i++) {$0=i; x=x+gsub("5",""); } print x} '

How to add multiple line of output one by one to a variable in Bash?

This might be a very basic question but I was not able to find solution. I have a script:
If I run w | awk '{print $1}' in command line in my server I get:
f931
smk591
sc271
bx972
gaw844
mbihk988
laid640
smk59
ycc951
Now I need to use this list in my bash script one by one and manipulate some operation on them. I need to check their group and print those are in specific group. The command to check their group is id username. How can I save them or iterate through them one by one in a loop.
what I have so far is
tmp=$(w | awk '{print $1})
But it only return first record! Appreciate any help.
Populate an array with the output of the command:
$ tmp=( $(printf "a\nb\nc\n") )
$ echo "${tmp[0]}"
a
$ echo "${tmp[1]}"
b
$ echo "${tmp[2]}"
c
Replace the printf with your command (i.e. tmp=( $(w | awk '{print $1}') )) and man bash for how to work with bash arrays.
For a lengthier, more robust and complete example:
$ cat ./tstarrays.sh
# saving multi-line awk output in a bash array, one element per line
# See http://www.thegeekstuff.com/2010/06/bash-array-tutorial/ for
# more operations you can perform on an array and its elements.
oSET="$-"; set -f # save original set flags and turn off globbing
oIFS="$IFS"; IFS=$'\n' # save original IFS and make IFS a newline
array=( $(
awk 'BEGIN{
print "the quick brown"
print " fox jumped\tover\tthe"
print "lazy dogs back "
}'
) )
IFS="$oIFS" # restore original IFS value
set +f -$oSET # restore original set flags
for (( i=0; i < ${#array[#]}; i++ ));
do
printf "array[%d] of length=%d: \"%s\"\n" "$i" "${#array[$i]}" "${array[$i]}"
done
printf -- "----------\n"
printf -- "array[#]=\n\"%s\"\n" "${array[#]}"
printf -- "----------\n"
printf -- "array[*]=\n\"%s\"\n" "${array[*]}"
.
$ ./tstarrays.sh
array[0] of length=22: "the quick brown"
array[1] of length=23: " fox jumped over the"
array[2] of length=21: "lazy dogs back "
----------
array[#]=
"the quick brown"
array[#]=
" fox jumped over the"
array[#]=
"lazy dogs back "
----------
array[*]=
"the quick brown fox jumped over the lazy dogs back "
A couple of non-obvious key points to make sure your array gets populated with exactly what your command outputs:
If your command output can contain globbing characters than you should disable globbing before the command (oSET="$-"; set -f) and re-enable it afterwards (set +f -$oSET).
If your command output can contain spaces then set IFS to a newline before the command (oIFS="$IFS"; IFS=$'\n') and set it back to it's old value after the command (IFS="$oIFS").
tmp=$(w | awk '{print $1}')
while read i
do
echo "$i"
done <<< "$tmp"
You can use a for loop, i.e.
for user in $(w | awk '{print $1}'); do echo $user; done
which in a script would look nicer as:
for user in $(w | awk '{print $1}')
do
echo $user
done
You can use the xargs command to do this:
w | awk '{print $1}' | xargs -I '{}' id '{}'
With the -I switch, xargs will take each line of its standard input separately, then construct and execute a command line by replacing the specified string '{}' in the command line template with the input line
I guess you should use who instead of w. Try this out,
who | awk '{print $1}' | xargs -n 1 id

integer expression expected [bash does not understand .]

I made a small script to kill PID's if they exceed expected cpu usage. It works, but there is a small problem.
Script:
while [ 1 ];
do
cpuUse=$(ps -eo %cpu | sort -nr | head -1)
cpuMax=80
PID=$(ps -eo %cpu,pid | sort -nr | head -1 | cut -c 6-20)
if [ $cpuUse -gt $cpuMax ] ; then
kill -9 "$PID"
echo Killed PID $PID at the usage of $cpuUse out of $cpuMax
fi
exit 0
sleep 1;
done
It works if the integer is three digits long but fails if it drops to two and displays this:
./kill.sh: line 7: [: 51.3: integer expression expected
My question here is, how do I make bash understand the divider so it can kill processes under three digits.
You are probably getting leading space in that variable. Try piping with tr to strip all spaces first:
cpuUse=$(ps -eo %cpu | sort -nr | head -1 | tr -d '[[:space:]]')
Remove text after dot from cpuUse variable:
cpuUse="${cpuUse%%.*}"
Also better to use quotes in if condition:
if [ "$cpuUse" -gt "$cpuMax" ] ; then
OR better use arithmetic operator (( and )):
if (( cpuUse > cpuMax )); then
As you see, bash doesn't grok non-integer numbers. You need to eliminate the decimal point and the following digits from $cpuUse before doing the comparison":
cpuUse=$(sed 's/\..*/' <<<$cpuUse)
However, this is really a job for awk. It will simplify much of what you're doing. Whenever you find yourself with greps of greps, or head and then cuts, you should be dealing with awk. Awk can easily combine these multiple piped seds, greps, cuts, heads, into a single command.
By the way, the correct ps command is:
$ ps -eocpu="",pid=""
Using the ="" will eliminate the heading and simply give you the CPU and PID.
Looking at your program, there's no real need to sort. You're simply looking for all processes above that $cpuMax threshold:
ps -eo %cpu="",pid="" | awk '$1 > 80 {print $2}'
That prints out your PIDs which are over your threshold. Awk automatically loop through your entire input line-by-line. Awk also automatically divides each line into columns, and assigns each a variable from $1 and up. You can change the field divider with the -F parameter.
The above awk says look for all lines where the first column is above 80%, (the CPU usage) and print out the second column (the pid).
If you want some flexibility and be able to pass in different $cpuMax, you can use the -v parameter to set Awk variables:
ps -eo %cpu="",pid="" | awk -vcpuMax=$cpuMax '$1 > cpuMax {print $2}'
Now that you can pipe the output of this command into a while to delete all those processes:
pid=$(ps -eo %cpu="",pid="" | awk -vcpuMax=$cpuMax '$1 > cpuMax {print $2}')
if [[ -n $pid ]]
then
kill -9 $pid
echo "Killed the following processes:" $pid
fi

how to get the last login time for all users in one line for different shells

I can make the following line work on ksh
for user in $( awk -F: '{ print $1}' /etc/passwd); do last $user | head -1 ; done | tr -s "\n" |sort
But I'd like to make it work on UNIX sh and UNIX csh. (in linux sh it runs fine, but linux is not unix...)
I know there are limitations for this since it seems that each UNIX(*) has its own variations on the syntax.
update: sorry, there are some restrictions here:
I can't write on the disk, so I can't save scripts.
how do i write this in CSH?
This awk-script seems to be the equivalent to you loop above:
{
cmd = "last "$1
cmd | getline result
printf "%s", result
}
use it like this:
awk -F: -f script_above.awk /etc/passwd
Pipe the output to sort
As a one-liner:
$ awk -F: '{cmd = "last "$1; cmd | getline result;printf "%s", result}' /etc/passwd
This might do the trick for you, should be POSIX compliant:
last | awk 'FNR==NR{split($0,f,/:/);a[f[1]];next}($1 in a)&&++b[$1]==1' /etc/passwd - | sort
You don't really need Awk for this.
while IFS=: read user _; do
last "$user" | head -n 1
done </etc/passwd # | grep .
Instead of reinvent it in Csh, how about
sh -c 'while IFS=: read user _; do last "$user" | head -n 1; done </etc/passwd'
You will get empty output for users who have not logged in since wtmp was rotated; maybe add a | grep . to weed those out. (I added it commented out above.)
To reiterate, IFS=: sets the shell's internal field separator to a colon, so that read will split the password file on that.
Just use simple command:
lastlog

Bash escaping and syntax

I have a small bash file which I intend to use to determine my current ping vs my average ping.
#!/bin/bash
output=($(ping -qc 1 google.com | tail -n 1))
echo "`cut -d/ -f1 <<< "${output[3]}"`-20" | bc
This outputs my ping - 20 ms, which is the number I want. However, I also want to prepend a + if the number is positive and append "ms".
This brings me to my overarching problem: Bash syntax regarding escaping and such heavy "indenting" is kind of flaky.
While I'll be satisfied with an answer of how to do what I wanted, I'd like a link to, or explanation of how exactly bash syntax works dealing with this sort of thing.
output=($(ping -qc 1 google.com | tail -n 1))
echo "${output[3]}" | awk -F/ '{printf "%+fms\n", $1-20}'
The + modifier in printf tells it to print the sign, whether it's positive or negative.
And since we're using awk, there's no need to use cut or bc to get a field or do arithmetic.
Escaping is pretty awful in bash if you use the deprecated `..` style command expansion. In this case, you have to escape any backticks, which means you also have to escape any other escapes. $(..) nests a lot better, since it doesn't add another layer of escaping.
In any case, I'd just do it directly:
ping -qc 1 google.com.org | awk -F'[=/ ]+' '{n=$6}
END { v=(n-20); if(v>0) printf("+"); print v}'
Here's my take on it, recognizing that the result from bc can be treated as a string:
output=($(ping -qc 1 google.com | tail -n 1))
output=$(echo "`cut -d/ -f1 <<< "${output[3]}"`-20" | bc)' ms'
[[ "$output" != -* ]] && output="+$output"
echo "$output"
Bash cannot handle floating point numbers. A workaround is to use awk like this:
#!/bin/bash
output=($(ping -qc 1 google.com | tail -n 1))
echo "`cut -d/ -f1 <<< "${output[3]}"`-20" | bc | awk '{if ($1 >= 0) printf "+%fms\n", $1; else printf "%fms\n", $1}'
Note that this does not print anything if the result of bc is not positive
Output:
$ ./testping.sh
+18.209000ms

Resources