To keep my data in structured form I often end up having deep folder-trees where it can be a little annoying going back and forward in so I wrote a function to generate an html tree of the cwd:
function toc_date {
# Get the date in _YYYY-MM-DD format
D=`date +_%F`
# Build the tree, -H flag to output as HTML,
# . to use current dir in <a href> link
tree -H . >> toc$D.html
}
The problem arose when I wrote a follow up function to remove old folder-tree files, specifically in the first expr of [ "$(ls -b toc_* | wc -l)" -gt "0" ] which gives an error when no files are found even though it's value is correctly set to 0, which should make it skip the if statement (right?). The code works as intended, but error messages are usually not a sign of good code, so was hoping that someone might be able to suggest improvements?
function old_stuff {
# Number of lines i.e. files matching toc_*
if [ "$(ls -b toc_* | wc -l)" -gt "0" ]
then
# Show files/directories so we don't remove stuff unintended
ls toc_*
while true; do
read -p "Do you wish to remove old TOCs? [Y/n]" yn
case $yn in
[Nn]* ) break;;
[Yy]* ) rm toc_*; break;;
* ) echo "Please answer yes or no.";;
esac
done
fi
# Go ahead and generate the html tree
toc_date
}
Change it to:
if [ "$(ls -b toc_* 2>/dev/null | wc -l)" -gt "0" ]
Another slightly more readable and better way is (stripped off unnecessary quotes):
if [ $(ls | grep -c '^toc_') -gt 0 ]
just an advice , take this out"$(ls -b toc_* | wc -l)" and set it inside a variable
fileCount=$(ls -b toc_* 2>/dev/null | wc -l)
The 2>/dev/null is to send stderr i.e any kind of error message to /dev/null
And also it is advisable you always use [[ ]] whenever you are writing an if statement, to avoid errors like unary operator is expected
Related
I am writing a shell script to check two things at one time. The first condition is to check for the existence of a specific file and the second condition is to confirm that there is only one file in that directory.
I am using the following code:
conf_file=ls -1 /opt/files/conf.json 2>/dev/null | wc -l
total_file=ls -1 /opt/files/* 2>/dev/null| wc -l
if [ $conf_file -eq 1 ] && [ $total_file -eq 1 ]
then
echo "done"
else
echo "Not Done"
fi
It is returning the following error
0
0
./ifexist.sh: 4: [: -eq: unexpected operator
Not Done
I am probably doing a very silly mistake. Can anyone help me a little bit?
One of the reasons you should normally not parse ls is that you can get strange results when you have files with newlines. In your case that won't be an issue, because any file different from json.conf should make the test fail. However you should make the code counting the files be future-proof. You can use find for this.
Your code can be changed into
jsonfile="/opt/files/conf.json"
countfiles=$(find /opt/files -maxdepth 1 -type f -exec printf '.\n' \; | wc -l)
if [[ -f "${jsonfile}" ]] && (( "${countfiles}" == 1)); then
echo "Done"
else
echo "Not Done"
fi
When you say this:
conf_file=ls -1 /opt/files/conf.json 2>/dev/null | wc -l
That assigns the value "ls" to the variable conf_file, and then tries to run a command called "-1" and pipe the result to wc If you want to run a pipe sequence, you have to enclose it in $( ):
conf_file=$(ls -1 /opt/files/conf.json 2./dev/null | wc -l)
Next, when combining clauses in the test command ([), do it inside the command:
if [ $conf_file -eq 1 -a $total_file -eq 1 ]
However, there are better ways to do this. You can check if a file exists with "-f", and you can just check whether the output of ls matches what you expect, without creating variables or running other commands:
if [ -f /opt/files/conf.json -a "$(ls /opt/files/conf.*)" -eq "/opt/files/conf.json" ]
However, it is not a friendly practice to prohibit other files. In many cases, people might want to leave backup or test copies (conf.json.bak or conf.json.test), and there's no reason for you to block that.
I have written a script to check process is running or not,it work fine but while testing it, i have found that it not include top command count running in other terminal
check-process.sh
#!/bin/sh
OK=1
CRITICAL=0
PROCESS_NUM=$( ps -ef | grep $1 | grep -v "grep "|grep -v "sh"|wc -l )
#echo $PROCESS_NUM
if [ $PROCESS_NUM = $OK ]
then
echo "OK"
elif [ $PROCESS_NUM = $CRITICAL ]
then
echo "CRITICAL"
elif [ $PROCESS_NUM > $OK ]
then
echo "MULTIPLE process are runing"
else
echo "error"
fi
And i run top command in two terminals and run this script as follow:
./check-process.sh top
and out put is 0 CRITICAL , but when i run normal command ps -ef |grep -v "grep "| wc -l it gives two counts.
That mess of greps just has to go.
One "trick" for finding processes by name without finding your grep is to use a regular expression. That is, after all, what the Global Regular Expression Print command is for. You can use parameter expansion to construct a safe regular expression based on your input string, perhaps like this:
#!/bin/sh
if [ -z "$1" ]; then
echo "I'd love me an option." >&2
exit 1
fi
OK=1
CRITICAL=0
x="${1#?}" # make a temporary string missing the 1st chararcter,
re="[${1%$x}]$x" # then place the 1st character within square brackets.
PROC_COUNT=$( ps -ef | grep -w -c "$re" ) # yay, just one pipe.
if [ "$PROC_COUNT" -eq "$OK" ]; then
echo "OK"
elif [ "$PROC_COUNT" -eq "$CRITICAL" ]; then
echo "CRITICAL"
elif [ "$PROC_COUNT" -gt "$OK" ]; then
echo "MULTIPLE process are running"
else
echo "error"
fi
There are a few notable changes here:
I added something to fail with better explanation if no option is given.
The pipeline, of course. And the lines that create $re.
We're using -gt and -eq to test numeric values. man test for details.
I renamed your count variable to be clearer. What is a "PROCESS_NUM" really? Sounds like a PID to me.
All variables are quoted. I don't need to tell you why, you have the Google.
That said, you should also consider using pgrep instead of any sort of counting pipe, if it's available on your system. Try running pgrep and see what your OS tells you.
I'm having an issue with tail & grep in shell script if statement. If I run tail -5 mylog.log | grep -c "Transferred: 0" in shell, it runs as it should, but in this shell script if statement:
# Parse log for results
if [ tail -1 "$LOGFILE" | grep -c "Failed" ] ; then
RESULT=$(tail -1 "$LOGFILE")
elif [ tail -5 "$LOGFILE" | grep -c "Transferred: 0" ] ; then
RESULT=""
else
RESULT=$(tail -5 "$LOGFILE")
fi
I get
... [: missing `]'
grep: ]: No such file or directory
for both of the grep lines.
It's clearly to do with the closing ] being seen as part of the grep (and thus missing) but I'm using the correct whitespace so I can't figure out what's going on? What am I doing wrong here?
Thanks,
PJ
Immediate Issue / Immediate Fix
Take out the brackets.
if tail -1 "$logfile" | grep -q "Failed" ; then
[ is not part of if syntax. Rather, it's a synonym for the command named test (which is typically both available as a shell builtin and an external binary, like /bin/test or /usr/bin/test).
Thus, your original code was running [ tail -1 "$logfile", and piping its result to grep -q "Failed" ]. The first [ was failing because it didn't see an ending ] -- which is mandatory when invoked by that name rather than with the name test -- and also because its parameters weren't a test it knew how to parse; and the second grep didn't know what the ] it was being piped meant, trying to find a file by that name.
Other Notes
Try to run external commands -- like tail -- as little as possible. There's a very significant startup cost.
Consider the following, which runs tail only once:
#!/bin/bash
# ^^^^- IMPORTANT: bash, not /bin/sh
last_5_lines="$(tail -5 "$logfile")"
last_line="${last_5_lines##*$'\n'}"
if [[ $last_line = *Failed* ]]; then
result=$last_line
elif [[ $last_5_lines =~ 'Transferred:'[[:space:]]+'0' ]]; then
result=''
else
result=$last_5_lines
fi
I am looking to create a shell script to read the message log and when finds the correct string perform an action. So far I have the following:
#!/bin/bash
string="ntp engine ready"
tail -n 0 -f /var/log/messages | \
while read LINE
do
echo "$LINE | grep -q $string"
if [ $? == 0];then
shttpclient "http://127.0.0.1/do/action"
fi
done
But, I get the following error:
grep: engine: No such file or directory
grep: ready: No such file or directory
Even when I see the logger has outputted ntp engine ready.
Firstly, you need to fix your quotes:
echo "$LINE" | grep -q "$string"
Secondly, you can simply do:
if echo "$LINE" | grep -q "$string"; then
rather than checking the return code $? manually. Remember that [ is a command too and if is just checking its return code.
If you do need to use [, remember that ] is an argument to the command so it is essential to surround it with spaces:
if [ $? = 0 ]
I have also removed the second = as it is a bash extension to support it. Actually you are doing an integer comparison, so really it should be one of the following:
if [ $? -eq 0 ] # POSIX compliant
if (( $? == 0 )) # bash arithmetic context
Alter the line as follows:
echo "$LINE" | grep -q "$string"
The quotes were not set correctly. Like when you execute that: grep -q ntp engine ready; ntp is the string to search and engine and ready are the files. It must look like: grep -q "ntp engine ready".
So I have been struggling with this task for eternity and still don't get what went wrong. This program doesn't seem to download ANY pdfs. At the same time I checked the file that stores final links - everything stored correctly. The $PDFURL also checked, stores correct values. Any bash fans ready to help?
#!/bin/sh
#create a temporary directory where all the work will be conducted
TMPDIR=`mktemp -d /tmp/chiheisen.XXXXXXXXXX`
echo $TMPDIR
#no arguments given - error
if [ "$#" == "0" ]; then
exit 1
fi
# argument given, but wrong format
URL="$1"
#URL regex
URL_REG='(https?|ftp|file)://[-A-Za-z0-9\+&##/%?=~_|!:,.;]*[-A-Za-z0-9\+&##/%=~_|]'
if [[ ! $URL =~ $URL_REG ]]; then
exit 1
fi
# go to directory created
cd $TMPDIR
#download the html page
curl -s "$1" > htmlfile.html
#grep only links into temp.txt
cat htmlfile.html | grep -o -E 'href="([^"#]+)\.pdf"' | cut -d'"' -f2 > temp.txt
# iterate through lines in the file and try to download
# the pdf files that are there
cat temp.txt | while read PDFURL; do
#if this is an absolute URL, download the file directly
if [[ $PDFURL == *http* ]]
then
curl -s -f -O $PDFURL
err="$?"
if [ "$err" -ne 0 ]
then
echo ERROR "$(basename $PDFURL)">&2
else
echo "$(basename $PDFURL)"
fi
else
#update url - it is always relative to the first parameter in script
PDFURLU="$1""/""$(basename $PDFURL)"
curl -s -f -O $PDFURLU
err="$?"
if [ "$err" -ne 0 ]
then
echo ERROR "$(basename $PDFURLU)">&2
else
echo "$(basename $PDFURLU)"
fi
fi
done
#delete the files
rm htmlfile.html
rm temp.txt
P.S. Another minor problem I have just spotted. Maybe the problem is with the if in regex? I pretty much would like to see something like that there:
if [[ $PDFURL =~ (https?|ftp|file):// ]]
but this doesn't work. I don't have unwanted parentheses there, so why?
P.P.S. I also ran this script on URLs beginning with http, and the program gave the desired output. However, it still doesn't pass the test.