The shell script does not execute the removing the files - bash

I have the below script for removing the web cache and files from /tmp and /var/tmp directory to bring down the disk utilization percentage. When I execute the script it doesn't remove the files and gives nothing.
#!/bin/bash
#set -x
#THRESHOLD=30
TEMP1=$(cd /tmp)
TEMP2=$(cd /var/tmp)
df -h | grep -v '^Filesystem' | awk '{print $1,$5}' | while read output;
do
used=$(echo $output | awk '{print $5}' | cut -d '%' -f1)
diskpart=$(echo $output | awk '{print $2}')
if [[ "$used" -ge "30" ]]; then
echo "Disk utilization percentage is more"
# sync; echo 1 > /proc/sys/vm/drop_caches
rm -rf "$TEMP1/*"
rm -rf "$TEMP2/*"
fi
done
Output below:
srikant#ubuntu:~$ ./example.sh
srikant#ubuntu:~$ ./example.sh
srikant#ubuntu:~$
The df -h command output:
srikant#ubuntu:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 393M 12M 382M 3% /run
/dev/sda1 19G 9.5G 8.2G 54% /
tmpfs 2.0G 272K 2.0G 1% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 393M 0 393M 0% /run/user/121
tmpfs 393M 52K 393M 1% /run/user/1000
I tried with sudo before rm -rf and while executing the script also. But still it is not working.
Please help.

When you generate output it only has two fields in it, the original $1 and $5 in the first awk '{print $1,$5}', for example /dev/sda1 and 54%. Then in:
used=$(echo $output | awk '{print $5}' | cut -d '%' -f1)
you only have these two fields, so the awk '{print $5}' part gives an empty string. It should be:
used=$(echo $output | awk '{print $2}' | cut -d '%' -f1)
In addition to that:
TEMP1=$(cd /tmp)
TEMP2=$(cd /var/tmp)
will be empty strings, because cd does not write anything to standard output. This should be:
TEMP1='/tmp'
TEMP2='/var/tmp'
The quotes are optional in this case.
Also:
rm -rf "$TEMP1/*"
will look for a literal file called *, since filename expansion is not done inside quotes!
EDIT: My preferred solution would be to loose the external programs awk and cut:
temp1='/tmp' # Avoid uppercase variable names
temp2='/var/tmp' # they can collide with shell names
while read -r diskpart Size Used Avail Use Mounted
do
# Remove the trailing %
used=${Use%%%*}
# A neater from of numeric comparison
if (( used >= 30 ))
then
echo "Disk utilization percentage is more"
#sync; echo 1 > /proc/sys/vm/drop_caches
# Note that putting * inside quotes won't be expanded
rm -rf "$temp1"/*
rm -rf "$temp2"/*
fi
done < <(df -h)

Related

Named pipe swallowing first field of Linux command output

I'm trying to parse the output of the Linux df tool for use in a machine status report. I'm using almost identical code to parse the output of the ps tool and it works fine (all fields are available in read loop) but in the code below the first field output from awk (percentUsed) is missing when I read from the named pipe.
#!/bin/sh
mkfifo dfPipe0
IFS=" "
df -h | awk '{ print $6" "$3" "$7" "$1 }' > dfPipe0 &
while read -r percentUsed size mountedOn fileSystem
do
printf "%s\n" "${percentUsed} | ${size} | ${mountedOn} | ${fileSystem}"
done < dfPipe0
rm dfPipe0
Sample df + awk output
$ df -h | awk '{ print $6" "$3" "$7" "$1 }'
Use% Size Mounted Filesystem
0% 1.9G /dev devtmpfs
- 0 /sys/kernel/security securityfs
4% 1.9G /dev/shm tmpfs
$
I edited my code to use the standard pipe suggestion by #Barmar and it resolves my problem. The code below works fine.
df -h | \
while read -r fileSystem size used avail percentUsed mountedOn
do printf "%s\n" "$fileSystem | $size | $used | $avail | $percentUsed | $mountedOn"
done

BASH Free Space Script: Variation

I made a script in order to check the available space on root, /var, /tmp, /usr and /opt since I need 1.5G free on each of them. Of course, there are times when all those are in root, not in another partitions.
But I got some error from my script:
1- Sometimes the output its weird when it runs in a non-English System.
2- --output in df -h its not available since some systems are old.
I want to change this script a little bit since I want it to work in every machine and I would like to ask for ideas.
I have tested the script on 650 machines already. 140 of them created the problems that I mention.
printf "root=" ; df -h --output=avail / | grep -v Avail | xargs | tr -d '[:space:]'
printf ",opt=" ; df -h --output=avail /opt | grep -v Avail | xargs | tr -d '[:space:]'
printf ",usr=" ; df -h --output=avail /usr | grep -v Avail | xargs | tr -d '[:space:]'
printf ",tmp=" ; df -h --output=avail /tmp | grep -v Avail | xargs | tr -d '[:space:]'
printf ",var=" ; df -h --output=avail /var | grep -v Avail | xargs
1- Problem:
Wrong: root=Dispo 1.5G,var=Dispo 1.5G,usr=Dispo 1.5G,tmp=Dispo
1.5G,opt=Dispo 1.5G
Correct: root=1.5G,var=1.5G,usr=1.5G,tmp=1.5G,opt=1.5G
2- Problem:
df: Unbekannte Option »--output=avail«
„df --help“ gibt weitere Informationen.
root= opt= usr= tmp= var=
EDIT:
Looks like #CharlesDufy answer worked. I made the script like this:
#!/bin/bash
sudo df -h / /var /tmp /opt /usr > freespace.txt
rootSpace=$(awk "NR==2 { print $4 }" freespace.txt)
varSpace=$(awk "NR==3 { print $4 }" freespace.txt)
tmpSpace=$(awk "NR==4 { print $4 }" freespace.txt)
optSpace=$(awk "NR==5 { print $4 }" freespace.txt)
usrSpace=$(awk "NR==6 { print $4 }" freespace.txt)
customProp="root=$rootSpace,var=$varSpace,tmp=$tmpSpace,opt=$optSpace,usr=$usrSpace"
#and the rest....
Also I want to ask 2 more questions:
Is it possible to take the freespace.txt output without creating a file?
If the 4 folders are in root, like they are not separte partitions, my output looks like this
root=12G,var=12G,tmp=12G,opt=12G,usr=12G
Is there a way to make an output like this without tons of IF, ELIF or ELSE comands?
root=12G,var=root,tmp=root,opt=root,usr=root

Mailing out text file in table format using shell script

I am also looking for the same,
My text file have df -h information of my server,need to send the detail to my mail in table format.
Use% Avail Used Filesystem
47% 28G 24G rootfs
0% 11M 0 udev
1% 396M 160k tmpfs
47% 28G 24G /dev/disk/
0% 5.3M 0 tmpfs
1% 793M 13k tmpfs
2% 50G 520M /dev/xvdg
24% 39G 12G /dev/xvdf
I need table format
as
enter image description here
I have tried your steps but getting mail as below
input="/home/infrarunsvc/output.txt"
tmpfile="/home/infrarunsvc/tmpfile.html"
echo 'Content-Type: text/html; charset="us-ascii" ' > "$tmpfile"
awk 'BEGIN{print "<html><body><table border=1>"} {print "<tr>";for(i=1;i<=NF;i++)print "<td>" $i"</td>";print "</tr>"} END{print "</table></body></html>"}' "$input" >> "$tmpfile"
mail -s "test" xxxxxx.com < "$tmpfile"
Kindly help me in formatting it.
you can try something like this;
#!/bin/bash
df -h > /home/infrarunsvc/output.txt
tmpfile="/home/infrarunsvc/tmpfile.html"
input="/home/infrarunsvc/output.txt"
echo '<html><body><table border=1>' > "$tmpfile"
awk '{print "<tr>";for(i=1;i<=NF;i++)print "<td>" $i"</td>";print "</tr>"} END{print "</table></body></html>"}' "$input" >> "$tmpfile"
cat $tmpfile | mail -s "$(echo -e "Test\nContent-Type: text/html")"
yourmailaddress#...com

bash script that monitor a disk partition's usage

df shows
-bash-4.1# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 1918217320 1783986384 36791092 98% /
tmpfs 16417312 0 16417312 0% /dev/shm
/dev/sda1 482214 148531 308784 33% /boot
/dev/sdb1 1922858352 1373513440 451669312 76% /disk2
I need to bash script a function that returns 1 if an paritions become 100% full.
how can this be done? what commands can I use to parse out the output of df?
This should do it:
disks_space() {
! df -P | awk '{print $5}' | grep -Fqx '100%'
}
In other words, check if any of the lines in the fifth column of the POSIX df output contains the exact string "100%".
Probelm with percentage is if its a terrabyte disk 95% of that may still be lots of free gig - refer to the bottom script for actual disk space - the format 100 at the end of the example shows alert when it is below 100MB left on a partition
diskspace.sh
#!/bin/sh
# set -x
# Shell script to monitor or watch the disk space
# It will send an email to $ADMIN, if the (free available) percentage of space is >= 90%.
# -------------------------------------------------------------------------
# Set admin email so that you can get email.
ADMIN="root"
# set alert level 90% is default
ALERT=90
# Exclude list of unwanted monitoring, if several partions then use "|" to separate the partitions.
# An example: EXCLUDE_LIST="/dev/hdd1|/dev/hdc5"
EXCLUDE_LIST="/auto/ripper"
#
#::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
#
function main_prog() {
while read output;
do
echo $output
usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1)
partition=$(echo $output | awk '{print $2}')
if [ $usep -ge $ALERT ] ; then
if [ "$partition" == "/var" ]; then
# echo "Running out of space \"$partition ($usep%)\" on server $(hostname), $(date)"
echo "Running out of space \"$partition ($usep%)\" on server $(hostname), $(date)" | mail -s "Alert: Almost out of disk space $usep%" $ADMIN
# Extra bits you may wish to do -
#for FILE in `find $partition -size +1G -print`
#do
# echo $FILE
# DATE=`date +%Y-%m-%d_%H%M`
# filename=`echo ${FILE##*/}`
# mkdir /mnt/san/$hostname
# echo cp $FILE /mnt/san/$(hostname)/$filename-$DATE
# #echo > $FILE
#done
fi
fi
done
}
if [ "$EXCLUDE_LIST" != "" ] ; then
df -hP | grep -vE "^[^/]|tmpfs|cdrom|${EXCLUDE_LIST}" | awk '{print $5 " " $6}' | main_prog
else
df -hP | grep -vE "^[^/]|tmpfs|cdrom"| awk '{print $5 " " $6}' | main_prog
fi
Or you could use this style of check I put in place for nagios (using snmp to connect to a remote host)
snmp_remote_disk_auto
#!/bin/bash
# This script takes:
# <host> <community> <megs>
snmpwalk="/usr/bin/snmpwalk"
snmpget="/usr/bin/snmpget"
function usage() {
echo "$0 localhost public 100"
echo "where localhost is server"
echo "public is snmp pass"
echo "100 is when it reaches below a 100Mb"
echo "-----------------------------------"
echo "define threshold below limit specific for partitions i.e. boot can be 50mb where as /var I guess we want to catch it at around 1 gig so"
echo "$0 localhost public 1024"
}
server=$1;
pass=$2
limit=$3;
errors_found="";
partitions_found="";
lower_limit=10;
graphtext="|"
if [ $# -lt 3 ]; then
usage;
exit 1;
fi
# takes <size> <used> <allocation>
calc_free() {
echo "$1 $2 - $3 * 1024 / 1024 / p" | dc
}
for partitions in $($snmpwalk -v2c -c $pass -Oq $server hrStorageDescr|grep /|egrep -v "(/mnt|/home|/proc|/sys)"|awk '{print $NF}'); do
if [[ $partitions =~ /boot ]]; then
limit=$lower_limit;
fi
if result=$($snmpwalk -v2c -c $pass -Oq $server hrStorageDescr | grep "$partitions$"); then
index=$(echo $result | sed 's/.*hrStorageDescr//' | sed 's/ .*//')
args=$($snmpget -v2c -c $pass -Oqv $server hrStorageSize$index hrStorageUsed$index hrStorageAllocationUnits$index | while read oid j ; do printf " $oid" ; done)
free=$(calc_free$args)
back_count=$(echo $partitions|grep -o "/"|wc -l)
if [[ $back_count -ge 2 ]]; then
gpartition=$(echo "/"${partitions##*/})
else
gpartition=$partitions;
fi
if [ "$free" -gt "$limit" ]
then
graphtext=$graphtext$gpartition"="$free"MB;;;0 "
#graphtext=$graphtext$partitions"="$free"MB;;;0 "
partitions_found=$partitions_found" $partitions ($free MB)"
else
graphtext=$graphtext$gpartition"="$free"MB;;;0 "
#graphtext=$graphtext$partitions"="$free"MB;;;0 "
errors_found=$errors_found" $partitions ($free MB)"
fi
else
graphtext=$graphtext$gpartition"="0"MB;;;0 "
#graphtext=$graphtext$partitions"="0"MB;;;0 "
errors_found=$errors_found" $paritions does_not_exist_or_snmp_is_not_responding"
fi
done
if [ "$errors_found" == "" ]; then
echo "OK: $partitions_found$graphtext"
exit 0
else
echo "CRITICAL: $errors_found$graphtext";
exit 2;
fi
./snmp_remote_disk_auto localhost public 100
OK: / (1879 MB) /var (2281 MB) /tmp (947 MB) /boot (175 MB)|/=1879MB;;;0 /var=2281MB;;;0 /tmp=947MB;;;0 /boot=175MB;;;0
Not a huge fan of excessive greps and awks as it can really bring errors over time..
I would just get the information for the folders that matter. Below is a sample of using stat which will give you the available BYTES in a folder, then converts it to MB (10**6). I roughly tested this on my RHEL6.x system.
folder_x_mb=$(($(stat -f --format="%a*%s" /folder_x)/10**6))
folder_y_mb=$(($(stat -f --format="%a*%s" /folder_y)/10**6))
folder_z_mb=$(($(stat -f --format="%a*%s" /folder_z)/10**6))
How about something like:
df | perl -wne 'if(/(\d+)%\s+(.*)/){print "$2 at $1%\n" if $1>90}'
You can change the threshold and instead of printing you can just exit:
df | perl -wne 'if(/(\d+)%\s+(.*)/){exit 1 if $1>99}'
Here is a simple script to check if there are already disk that reached their maximum capacity and -- if there were it would return / output 1.
#!/bin/sh
CHECK=$(df -Ph | grep '100%' | xargs echo | cut -d' ' -f5)
if [ "$CHECK" == "100%"]
then
echo 1
else
echo 0
fi
Try this: df -Ph | grep -v "Use%" | sed 's/%//g' | awk '$5 > LIMIT {print $1,$2,$3,$4,$5"%";}' | column -t'
It will return all df -Ph entries that exceed the LIMIT
For example, on my workstation, df -Ph returns:
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p1 92G 32G 56G 37% /
shmfs 98G 304K 98G 1% /dev/shm
192.168.1.1:/apache_cache 2.7T 851G 1.9T 32% /media/backup
/dev/dm-4 50G 49G 1.1G 98% /lun1
/dev/dm-7 247G 30G 218G 12% /lun2
Let's say I want to list the mount points that exceed 20% of capacity.
I use df -Ph | grep -v "Use%" | sed 's/%//g' | awk '$5 > 20 {print $1,$2,$3,$4,$5"%";}' | column -t, and it returns the following:
/dev/cciss/c0d0p1 92G 32G 56G 37% /
192.168.1.1:/apache_cache 2.7T 851G 1.9T 32% /media/backup
/dev/dm-4 50G 49G 1.1G 98% /lun1
The column -t part is here purely for the output to be readable.

Nested pipe output in function

I am trying to make a script that will output system information using several functions that call each other. Can someone tell me what's wrong with how the piped commands' I/O is handled?
#!/bin/bash
function lyellow() {
lyellow="$1"
echo -e -n "\033[0;33m$lyellow"
echo -e -n '\033[0m \n'
}
function red() {
RED="$1"
echo -e -n "\033[0;31m$RED"
echo -e -n '\033[0m \n'
}
function lcyan() {
LCYAN="$1"
echo -e -n "\033[0;36m$LCYAN"
echo -e -n '\033[0m \n'
}
function lgreen() {
LGREEN="$1"
echo -e -n "\033[1;32m$LGREEN"
echo -e -n '\033[0m \n'
}
field ()
{
HEADER="$1"
SUB1="$2"
COM1="$3"
SUB2="$4"
COM2="$5"
echo -e "$(red "$(echo -e "### $HEADER ###")")"
echo -e "$(lyellow "$(echo -e "$SUB1")")\n$(lcyan "$(echo -e "$($COM1)")")"
echo -e "$(lyellow "$(echo -e "$SUB2")")\n$(lcyan "$(echo -e "$($COM2)")")"
}
#set -x pipefail
SEP=$(seq -s= 40|tr -d '[:digit:]')
echo $SEP
echo -e "$(lgreen "$(hostname -f) :: $(hostname -i)")"
echo $SEP
#OS
field "Operating System" \
"Kernel:" "/bin/uname -srp" \
"Release:" "cat /etc/redhat-release"
echo $SEP
#DISK
field "Storage Devices" \
"Mounted Devices:" "mount|column -t" \
"Disk Free:" "df -kh|column -t"
echo $SEP
#Example
lcyan "$(echo -e "$(df -kh | column -t)")"
exit 0
The output from the "#OS" "field" call works. But the "#DISK" call doesn't like the pipes to "column -t". Under "#Example" the color function calls a literal piped "column -t" fine. Here is what the output looks like:
[root#CLFT1Q ~]# sh sysinfo.sh
=======================================
CLFT1Q.local :: 10.9.19.70
=======================================
### Operating System ###
Kernel:
Linux 2.6.18-348.3.1.el5 i686
Release:
Red Hat Enterprise Linux Server release 5.9 (Tikanga)
=======================================
### Storage Devices ###
sysinfo.sh: line 36: /bin/mount|column: No such file or directory
Mounted Devices:
df: invalid option -- |
Try `df --help' for more information.
Disk Free:
=======================================
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgsystem-lv_root
3.9G 3.3G 421M 89% /
/dev/mapper/vgsystem-lv_var
4.9G 2.3G 2.4G 49% /var
/dev/mapper/vgsystem-ora
3.0G 1008M 1.9G 36% /ora
/dev/sda1 99M 25M 69M 27% /boot
tmpfs 1014M 0 1014M 0% /dev/shm
clnsa05:/vol/ftpnfsqa1/ftp
29G 25G 4.2G 86% /ftp
Change "$($COM1)" to "$(eval "$COM1")", and similarly for $COM2. Variable expansions are only scanned for word splitting and wildcard expansion, not for command metacharacters like pipelines. You need to use eval to process it recursively as a command line.

Resources