I'm trying to parse the output of the Linux df tool for use in a machine status report. I'm using almost identical code to parse the output of the ps tool and it works fine (all fields are available in read loop) but in the code below the first field output from awk (percentUsed) is missing when I read from the named pipe.
#!/bin/sh
mkfifo dfPipe0
IFS=" "
df -h | awk '{ print $6" "$3" "$7" "$1 }' > dfPipe0 &
while read -r percentUsed size mountedOn fileSystem
do
printf "%s\n" "${percentUsed} | ${size} | ${mountedOn} | ${fileSystem}"
done < dfPipe0
rm dfPipe0
Sample df + awk output
$ df -h | awk '{ print $6" "$3" "$7" "$1 }'
Use% Size Mounted Filesystem
0% 1.9G /dev devtmpfs
- 0 /sys/kernel/security securityfs
4% 1.9G /dev/shm tmpfs
$
I edited my code to use the standard pipe suggestion by #Barmar and it resolves my problem. The code below works fine.
df -h | \
while read -r fileSystem size used avail percentUsed mountedOn
do printf "%s\n" "$fileSystem | $size | $used | $avail | $percentUsed | $mountedOn"
done
Related
I made a script in order to check the available space on root, /var, /tmp, /usr and /opt since I need 1.5G free on each of them. Of course, there are times when all those are in root, not in another partitions.
But I got some error from my script:
1- Sometimes the output its weird when it runs in a non-English System.
2- --output in df -h its not available since some systems are old.
I want to change this script a little bit since I want it to work in every machine and I would like to ask for ideas.
I have tested the script on 650 machines already. 140 of them created the problems that I mention.
printf "root=" ; df -h --output=avail / | grep -v Avail | xargs | tr -d '[:space:]'
printf ",opt=" ; df -h --output=avail /opt | grep -v Avail | xargs | tr -d '[:space:]'
printf ",usr=" ; df -h --output=avail /usr | grep -v Avail | xargs | tr -d '[:space:]'
printf ",tmp=" ; df -h --output=avail /tmp | grep -v Avail | xargs | tr -d '[:space:]'
printf ",var=" ; df -h --output=avail /var | grep -v Avail | xargs
1- Problem:
Wrong: root=Dispo 1.5G,var=Dispo 1.5G,usr=Dispo 1.5G,tmp=Dispo
1.5G,opt=Dispo 1.5G
Correct: root=1.5G,var=1.5G,usr=1.5G,tmp=1.5G,opt=1.5G
2- Problem:
df: Unbekannte Option »--output=avail«
„df --help“ gibt weitere Informationen.
root= opt= usr= tmp= var=
EDIT:
Looks like #CharlesDufy answer worked. I made the script like this:
#!/bin/bash
sudo df -h / /var /tmp /opt /usr > freespace.txt
rootSpace=$(awk "NR==2 { print $4 }" freespace.txt)
varSpace=$(awk "NR==3 { print $4 }" freespace.txt)
tmpSpace=$(awk "NR==4 { print $4 }" freespace.txt)
optSpace=$(awk "NR==5 { print $4 }" freespace.txt)
usrSpace=$(awk "NR==6 { print $4 }" freespace.txt)
customProp="root=$rootSpace,var=$varSpace,tmp=$tmpSpace,opt=$optSpace,usr=$usrSpace"
#and the rest....
Also I want to ask 2 more questions:
Is it possible to take the freespace.txt output without creating a file?
If the 4 folders are in root, like they are not separte partitions, my output looks like this
root=12G,var=12G,tmp=12G,opt=12G,usr=12G
Is there a way to make an output like this without tons of IF, ELIF or ELSE comands?
root=12G,var=root,tmp=root,opt=root,usr=root
I have the below script for removing the web cache and files from /tmp and /var/tmp directory to bring down the disk utilization percentage. When I execute the script it doesn't remove the files and gives nothing.
#!/bin/bash
#set -x
#THRESHOLD=30
TEMP1=$(cd /tmp)
TEMP2=$(cd /var/tmp)
df -h | grep -v '^Filesystem' | awk '{print $1,$5}' | while read output;
do
used=$(echo $output | awk '{print $5}' | cut -d '%' -f1)
diskpart=$(echo $output | awk '{print $2}')
if [[ "$used" -ge "30" ]]; then
echo "Disk utilization percentage is more"
# sync; echo 1 > /proc/sys/vm/drop_caches
rm -rf "$TEMP1/*"
rm -rf "$TEMP2/*"
fi
done
Output below:
srikant#ubuntu:~$ ./example.sh
srikant#ubuntu:~$ ./example.sh
srikant#ubuntu:~$
The df -h command output:
srikant#ubuntu:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 393M 12M 382M 3% /run
/dev/sda1 19G 9.5G 8.2G 54% /
tmpfs 2.0G 272K 2.0G 1% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 393M 0 393M 0% /run/user/121
tmpfs 393M 52K 393M 1% /run/user/1000
I tried with sudo before rm -rf and while executing the script also. But still it is not working.
Please help.
When you generate output it only has two fields in it, the original $1 and $5 in the first awk '{print $1,$5}', for example /dev/sda1 and 54%. Then in:
used=$(echo $output | awk '{print $5}' | cut -d '%' -f1)
you only have these two fields, so the awk '{print $5}' part gives an empty string. It should be:
used=$(echo $output | awk '{print $2}' | cut -d '%' -f1)
In addition to that:
TEMP1=$(cd /tmp)
TEMP2=$(cd /var/tmp)
will be empty strings, because cd does not write anything to standard output. This should be:
TEMP1='/tmp'
TEMP2='/var/tmp'
The quotes are optional in this case.
Also:
rm -rf "$TEMP1/*"
will look for a literal file called *, since filename expansion is not done inside quotes!
EDIT: My preferred solution would be to loose the external programs awk and cut:
temp1='/tmp' # Avoid uppercase variable names
temp2='/var/tmp' # they can collide with shell names
while read -r diskpart Size Used Avail Use Mounted
do
# Remove the trailing %
used=${Use%%%*}
# A neater from of numeric comparison
if (( used >= 30 ))
then
echo "Disk utilization percentage is more"
#sync; echo 1 > /proc/sys/vm/drop_caches
# Note that putting * inside quotes won't be expanded
rm -rf "$temp1"/*
rm -rf "$temp2"/*
fi
done < <(df -h)
I am trying to write a script which empties file which is reaches 19% or more. I have the following code:
#!/bin/sh
ALERT="19" # setting alert to 19
df -h | grep /tmp | awk '{print $4}' | while read output; # reading the file system
do
echo $output
usep=$(echo $output | awk '{print $4}' | cut -d'%' -f1 )
if [ $usep -gt ALERT ]; then
cd /tmp
cat /dev/null > purge_foreign_mdn.log # filename
fi
done
I am getting this error:
purge_file.sh: line 10: [: -gt: unary operator expected
I tried -gt "19" and -gt '19'. Still getting different errors but at the same line.
You need to use $ALERT, not ALERT. As it is, you are comparing the $usep variable to the string ALERT, not the variable called $ALERT.
df shows
-bash-4.1# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 1918217320 1783986384 36791092 98% /
tmpfs 16417312 0 16417312 0% /dev/shm
/dev/sda1 482214 148531 308784 33% /boot
/dev/sdb1 1922858352 1373513440 451669312 76% /disk2
I need to bash script a function that returns 1 if an paritions become 100% full.
how can this be done? what commands can I use to parse out the output of df?
This should do it:
disks_space() {
! df -P | awk '{print $5}' | grep -Fqx '100%'
}
In other words, check if any of the lines in the fifth column of the POSIX df output contains the exact string "100%".
Probelm with percentage is if its a terrabyte disk 95% of that may still be lots of free gig - refer to the bottom script for actual disk space - the format 100 at the end of the example shows alert when it is below 100MB left on a partition
diskspace.sh
#!/bin/sh
# set -x
# Shell script to monitor or watch the disk space
# It will send an email to $ADMIN, if the (free available) percentage of space is >= 90%.
# -------------------------------------------------------------------------
# Set admin email so that you can get email.
ADMIN="root"
# set alert level 90% is default
ALERT=90
# Exclude list of unwanted monitoring, if several partions then use "|" to separate the partitions.
# An example: EXCLUDE_LIST="/dev/hdd1|/dev/hdc5"
EXCLUDE_LIST="/auto/ripper"
#
#::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
#
function main_prog() {
while read output;
do
echo $output
usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1)
partition=$(echo $output | awk '{print $2}')
if [ $usep -ge $ALERT ] ; then
if [ "$partition" == "/var" ]; then
# echo "Running out of space \"$partition ($usep%)\" on server $(hostname), $(date)"
echo "Running out of space \"$partition ($usep%)\" on server $(hostname), $(date)" | mail -s "Alert: Almost out of disk space $usep%" $ADMIN
# Extra bits you may wish to do -
#for FILE in `find $partition -size +1G -print`
#do
# echo $FILE
# DATE=`date +%Y-%m-%d_%H%M`
# filename=`echo ${FILE##*/}`
# mkdir /mnt/san/$hostname
# echo cp $FILE /mnt/san/$(hostname)/$filename-$DATE
# #echo > $FILE
#done
fi
fi
done
}
if [ "$EXCLUDE_LIST" != "" ] ; then
df -hP | grep -vE "^[^/]|tmpfs|cdrom|${EXCLUDE_LIST}" | awk '{print $5 " " $6}' | main_prog
else
df -hP | grep -vE "^[^/]|tmpfs|cdrom"| awk '{print $5 " " $6}' | main_prog
fi
Or you could use this style of check I put in place for nagios (using snmp to connect to a remote host)
snmp_remote_disk_auto
#!/bin/bash
# This script takes:
# <host> <community> <megs>
snmpwalk="/usr/bin/snmpwalk"
snmpget="/usr/bin/snmpget"
function usage() {
echo "$0 localhost public 100"
echo "where localhost is server"
echo "public is snmp pass"
echo "100 is when it reaches below a 100Mb"
echo "-----------------------------------"
echo "define threshold below limit specific for partitions i.e. boot can be 50mb where as /var I guess we want to catch it at around 1 gig so"
echo "$0 localhost public 1024"
}
server=$1;
pass=$2
limit=$3;
errors_found="";
partitions_found="";
lower_limit=10;
graphtext="|"
if [ $# -lt 3 ]; then
usage;
exit 1;
fi
# takes <size> <used> <allocation>
calc_free() {
echo "$1 $2 - $3 * 1024 / 1024 / p" | dc
}
for partitions in $($snmpwalk -v2c -c $pass -Oq $server hrStorageDescr|grep /|egrep -v "(/mnt|/home|/proc|/sys)"|awk '{print $NF}'); do
if [[ $partitions =~ /boot ]]; then
limit=$lower_limit;
fi
if result=$($snmpwalk -v2c -c $pass -Oq $server hrStorageDescr | grep "$partitions$"); then
index=$(echo $result | sed 's/.*hrStorageDescr//' | sed 's/ .*//')
args=$($snmpget -v2c -c $pass -Oqv $server hrStorageSize$index hrStorageUsed$index hrStorageAllocationUnits$index | while read oid j ; do printf " $oid" ; done)
free=$(calc_free$args)
back_count=$(echo $partitions|grep -o "/"|wc -l)
if [[ $back_count -ge 2 ]]; then
gpartition=$(echo "/"${partitions##*/})
else
gpartition=$partitions;
fi
if [ "$free" -gt "$limit" ]
then
graphtext=$graphtext$gpartition"="$free"MB;;;0 "
#graphtext=$graphtext$partitions"="$free"MB;;;0 "
partitions_found=$partitions_found" $partitions ($free MB)"
else
graphtext=$graphtext$gpartition"="$free"MB;;;0 "
#graphtext=$graphtext$partitions"="$free"MB;;;0 "
errors_found=$errors_found" $partitions ($free MB)"
fi
else
graphtext=$graphtext$gpartition"="0"MB;;;0 "
#graphtext=$graphtext$partitions"="0"MB;;;0 "
errors_found=$errors_found" $paritions does_not_exist_or_snmp_is_not_responding"
fi
done
if [ "$errors_found" == "" ]; then
echo "OK: $partitions_found$graphtext"
exit 0
else
echo "CRITICAL: $errors_found$graphtext";
exit 2;
fi
./snmp_remote_disk_auto localhost public 100
OK: / (1879 MB) /var (2281 MB) /tmp (947 MB) /boot (175 MB)|/=1879MB;;;0 /var=2281MB;;;0 /tmp=947MB;;;0 /boot=175MB;;;0
Not a huge fan of excessive greps and awks as it can really bring errors over time..
I would just get the information for the folders that matter. Below is a sample of using stat which will give you the available BYTES in a folder, then converts it to MB (10**6). I roughly tested this on my RHEL6.x system.
folder_x_mb=$(($(stat -f --format="%a*%s" /folder_x)/10**6))
folder_y_mb=$(($(stat -f --format="%a*%s" /folder_y)/10**6))
folder_z_mb=$(($(stat -f --format="%a*%s" /folder_z)/10**6))
How about something like:
df | perl -wne 'if(/(\d+)%\s+(.*)/){print "$2 at $1%\n" if $1>90}'
You can change the threshold and instead of printing you can just exit:
df | perl -wne 'if(/(\d+)%\s+(.*)/){exit 1 if $1>99}'
Here is a simple script to check if there are already disk that reached their maximum capacity and -- if there were it would return / output 1.
#!/bin/sh
CHECK=$(df -Ph | grep '100%' | xargs echo | cut -d' ' -f5)
if [ "$CHECK" == "100%"]
then
echo 1
else
echo 0
fi
Try this: df -Ph | grep -v "Use%" | sed 's/%//g' | awk '$5 > LIMIT {print $1,$2,$3,$4,$5"%";}' | column -t'
It will return all df -Ph entries that exceed the LIMIT
For example, on my workstation, df -Ph returns:
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p1 92G 32G 56G 37% /
shmfs 98G 304K 98G 1% /dev/shm
192.168.1.1:/apache_cache 2.7T 851G 1.9T 32% /media/backup
/dev/dm-4 50G 49G 1.1G 98% /lun1
/dev/dm-7 247G 30G 218G 12% /lun2
Let's say I want to list the mount points that exceed 20% of capacity.
I use df -Ph | grep -v "Use%" | sed 's/%//g' | awk '$5 > 20 {print $1,$2,$3,$4,$5"%";}' | column -t, and it returns the following:
/dev/cciss/c0d0p1 92G 32G 56G 37% /
192.168.1.1:/apache_cache 2.7T 851G 1.9T 32% /media/backup
/dev/dm-4 50G 49G 1.1G 98% /lun1
The column -t part is here purely for the output to be readable.
this is part of my code:
FCWWN=`fcinfo hba-port | grep HBA | awk '{print $4}'`
for i in ${FCWWN}
do
FCREMOTE=`fcinfo remote-port -p ${i} | grep Remote | nawk '{print $4}'`
PRODUCT=`/usr/sbin/luxadm display ${FCREMOTE} | grep Product | nawk -F: '{print $2}'`
CAPACITY=`/usr/sbin/luxadm display ${FCREMOTE} | grep capacity | nawk -F: '{print $2}'`
LUNNAME=`/usr/sbin/luxadm display ${FCREMOTE} | grep /dev/rdsk | grep -i -v DEVICE`
FCSTATE=`fcinfo hba-port ${i} | grep State: | nawk '{print $2}'`
echo ""
echo -e The FC Port WWN "\033[1m \E[36;40m ${i} \033[0m is \033[1m \E[37;42m ${FCSTATE} \033[0m"; echo -ne "\E[0m"
printf "################################################################################## \n"
printf "%-6s %9s %18s \n" "LUN:" "Storage" "Disk"
printf "%-6s %9s %18s \n" " " "Type" "Size"
printf "=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- \n"
printf "%-6s\n %9s\n %2s\n" "${LUNNAME}" "${PRODUCT}" "${CAPACITY}"
done
The output is :
The FC Port WWN 10000000c9822976 is online
##################################################################################
LUN: Storage Disk
Type Size
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/dev/rdsk/c4t60060E80102A8AF005119C37000000C2d0s2
/dev/rdsk/c4t60060E80102A8AF005119C37000000C0d0s2
/dev/rdsk/c4t60060E80102A8AF005119C370000016Cd0s2
/dev/rdsk/c4t60060E80102A8AF005119C3700000028d0s2
/dev/rdsk/c4t60060E80102A8AF005119C37000000E1d0s2
DF600F
DF600F
DF600F
DF600F
DF600F
131072.000 MBytes
131072.000 MBytes
65536.000 MBytes
262144.000 MBytes
65536.000 MBytes
And the desired output is :
The FC Port WWN 10000000c9822976 is online
##################################################################################
LUN: Storage Disk
Type Size
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/dev/rdsk/c4t60060E80102A8AF005119C37000000C2d0s2 DF600F 131072.000 MBytes
/dev/rdsk/c4t60060E80102A8AF005119C37000000C0d0s2 DF600F 131072.000 MBytes
/dev/rdsk/c4t60060E80102A8AF005119C370000016Cd0s2 DF600F 65536.000 MBytes
/dev/rdsk/c4t60060E80102A8AF005119C3700000028d0s2 DF600F 262144.000 MBytes
/dev/rdsk/c4t60060E80102A8AF005119C37000000E1d0s2 DF600F 65536.000 MBytes
Any help would be appreciate.
One quick way is to convert your three multi-line strings to arrays:
# bash 4 or later
mapfile -t LUN_ARR <<< "$LUNNAME"
mapfile -t PROD_ARR <<< "$PRODUCT"
mapfile -t CAP_ARR <<< "$CAPACITY"
# bash 3
IFS=$'\n' read -d '' -a LUN_ARR <<< "$LUNNAME"
IFS=$'\n' read -d '' -a PROD_ARR <<< "$PRODUCT"
IFS=$'\n' read -d '' -a CAP_ARR <<< "$CAPACITY"
Then simply iterate through the arrays with a C-style loop
for (( idx=0; idx < ${#LUN_ARR[#]}; idx++ )); do
printf "%-40s %9s %s\n" "${LUN_ARR[idx]}" "${PROD_ARR[idx]}" "${CAP_ARR[idx]}"
done
You're very close. I don't have resources to test, but I really thing this will solve your problem.
not
printf "%-6s\n %9s\n %2s\n" "${LUNNAME}" "${PRODUCT}" "${CAPACITY}"
but
printf "%-6s %9s %2s\n" "${LUNNAME}" "${PRODUCT}" "${CAPACITY}"
The extra '\n's will always put in a line breaks, right?
Also you could write this as one awk script.
OR at least reduce the number of procesess that you're starting by re-writting the front part of your script like
PRODUCT=`/usr/sbin/luxadm display ${FCREMOTE} | nawk -F: '/Product/{print $2}'`
And, as you're using nawk, that says solaris to me. Maybe you're writting /bin/sh (bourne shell scripts),
but if not, back-ticks have been deprecated since 1995 (at least). Use modern command-substitution like
PRODUCT=$(/usr/sbin/luxadm display ${FCREMOTE} | nawk -F: '/Product/{print $2}')
IHTH