There is some information that I am waiting for on a website. I do not wish to check it every hour. I want a script that will do this for me and notify me if this website has been updated with the keyword that I am looking for.
Here is a basic bash script for checking if the webpage www.nba.com contains the keyword Basketball. The script will output www.nba.com updated! if the keyword is found, if the keyword isn't found the script waits 10 minutes and checks again.
#!/bin/bash
while [ 1 ];
do
count=`curl -s "www.nba.com" | grep -c "Basketball"`
if [ "$count" != "0" ]
then
echo "www.nba.com updated!"
exit 0
fi
sleep 600
done
We don't want the site or the keyword hard-coded into the script, we can make these arguments with the following changes.
#!/bin/bash
while [ 1 ];
do
count=`curl -s "$1" | grep -c "$2"`
if [ "$count" != "0" ]
then
echo "$1 updated!"
exit 0
fi
sleep 600
done
Now to run the script we would type ./testscript.sh www.nba.com Basketball. We could change the echo command to have the script send an email or any other preferred way of notification. Note we should check that arguments are valid.
go and configure a google alert..
You can also crawl the website and search for the keyword you are interested in.
Related
I'm trying to find a way to use the while loop to check for a certain condition, and if this condition is met, another command will be executed.
For example:
while [ -z "$(ls -A test-dir)" ];
do
echo "directory is empty, checking again in 5 seconds"
sleep 5
done
This loop will end once the directory will have any files inside, I'm trying to find a proper way to execute another command at the end when the loop completed.
Figured out that I need to simply add the last command after the while:
while [ -z "$(ls -A test-dir)" ];
do
sleep 2
echo "sleeping 2 seconds"
done
echo "reached the end"
The last command will be executed only after the loop is done.
This is part of bigger code that checking the OS version and choosing the correct condition by it.
( after checking OS version, it will go to this if condition: )
if [ "$AK" = "$OS6" ]
then
if [ "$(ls -la /etc/init.d/discagent 2>/dev/null | wc -l)" == 1 ]
then
/etc/init.d/discagent restart 2>&1 > /dev/null
/etc/init.d/discagent status |& grep -qe 'running'
if [ $? -eq 0 ] ; then
echo " Done "
else
echo " Error "
fi
fi
fi
If im hashing service discagent restart the pipeline is passing.
But if im not hashing it then it hang and not given any errors, And on the output file it is showing only the second server (out of few) that its hang on, And not moving to the next server.
what could be the issue?
p. S
while running it direct on the server it is working.
Run this script manually on the servers, that this script will be running on.
You can use xtrace -x which would show each statement before being executed, when you use with -v with -x so this would be -xv and the statement would be outputted, and then the line after the output of the statement with the variables substituted before the code is substituted.
using -u would show you the exact error when this occurs.
Another command is the trap command to debug your bash script.
More information on the following webpage,
https://linuxconfig.org/how-to-debug-bash-scripts
My script is executed by Cron and every 2 min checks if xxx is running. If it is not in the process then the script will run it. The problem is that sometimes it runs it several times.
My problem is how to detect that the program is running several times?
How does bash detect that the pidof function returns several rather than one pid?
#!/bin/bash
PID=`pidof xxx`
if [ "$PID" = "" ];
then
cd
cd /home/pi
sudo ./xxx
echo "OK"
else
echo "program is running"
fi
You can use this script for doing the same. It will make sure script is executed once.
#!/bin/bash
ID=`ps -ef|grep scriptname|grep -v grep|wc -l`
if [ $ID -eq 0 ];
then
#run the script
else
echo "script is running"
fi
I have written a bash script that does not show any errors. However I would like to add conditional block list if success then show email success else show error message in email as shown in the code below.
scp -i id_rsa -r testuser#1.1.1.:/data1/scp ~/data/scp/files
success >> ~/data/scp/files/log.txt 2>&1
if success
then
| mail -s "Download
Successfull" abc#test.com <<< "Files Successfully Downloaded"
else
| mail -s "Error: Download Failed" abc#test.com <<< "Error File download
Failed!"
fi
Here is the working script without If else block
#!/module/for/bash
scp -i id_rsa -r test#1.1.1.1:/data1/scp ~/data/scp/files
echo success! >> ~/data/scp/files/log.txt 2>&1 | mail -s "Download
Successfull" abc#test.com <<< "Files Successfully
Downloaded" | mail -s "Error: Download Failed" abc#test.com <<<
"Error:file download Failed!"
The scp man page states: The scp utility exits 0 on success, and >0 if an error occurs.
So you can do something like:
if scp -i id_rsa -r testuser#1.1.1.:/data1/scp ~/data/scp/files
then
mail -s "Download Successful" abc#test.com <<<"Files Downloaded"
else
mail -s "Download Error" abc#test.com <<<"Download error"
fi
or
scp -i id_rsa -r testuser#1.1.1.:/data1/scp ~/data/scp/files
if [[ $? -eq 0 ]]
then
mail -s "Download Successful" abc#test.com <<<"Files Downloaded"
else
mail -s "Download Error" abc#test.com <<<"Download error"
fi
finally you may also want to look at something like storing the scp output. Use -q to have scp not print out progress meters and what not:
MYOUT=$(scp -q -i id_rsa -r testuser#1.1.1.:/data1/scp ~/data/scp/files 2>&1)
if [[ $? -eq 0 ]]
then
mail -s "Download Successful" abc#test.com <<<"$MYOUT"
else
mail -s "Download Error" abc#test.com <<<"$MYOUT"
fi
This link should clear the air. Hope it helped!
#Korthrun has already posted several ways to accomplish what I think you're trying to do; I'll take a look at what's going wrong in your current script. You seem to be confused about a couple of basic elements of shell scripting: pipes (|) and testing for command success/failure.
Pipes are used to pass the output of one command into the input of another (and possibly then chain the output of the second command into the input of a third command, etc). But when you use a pipe string like this:
echo success! >> ~/data/scp/files/log.txt 2>&1 |
mail -s "Download Successfull" abc#test.com <<< "Files Successfully Downloaded" |
mail -s "Error: Download Failed" abc#test.com <<< "Error:file download Failed!"
the pipes aren't actually doing anything. The first pipe tries to take the output of echo and feed it to the input of mail, but the >> in the echo command sends its output to a file instead, so no actual data is sent to the mail command. Which is probably good, because the <<< on the mail command tells it to ignore the regular input (from the pipe) and feed a string as input instead! Similarly, the second pipe tries to feed the output from the first mail command (there isn't any) to the last mail command, but again it's ignored due to another <<< input string. The correct way to do this is simply to remove the pipes, and run each command separately:
echo success! >> ~/data/scp/files/log.txt 2>&1
mail -s "Download Successfull" abc#test.com <<< "Files Successfully Downloaded"
mail -s "Error: Download Failed" abc#test.com <<< "Error:file download Failed!"
This is also causing a problem in the other version of your script, where you use:
if success
then
| mail -s "Download Successfull" abc#test.com <<< "Files Successfully Downloaded"
Here, there's no command before the pipe, so it doesn't make any sense at all (and you get a shell syntax error). Just remove the pipe.
Now, about success/failure testing: you seem to be using success as a command, but it isn't one. You can either use the command you want to check the success of directly as the if conditional:
if scp ...; then
echo "It worked!"
else
echo "It failed!"
fi
or use the shell variable $? which returns the exit status of the last command (success=0, failure=anything else):
scp ...
if [ $? -eq 0 ]; then
...
There's a subtlety here that's easy to miss: the thing after if is a command, but in the second form it appears to be a logical expression (testing whether $? is equal to 0). The secret is that [ is actually a command that evaluates logical expressions and then exits with success or failure depending on whether the expression was true or false. Do not mistake [ ] for some sort of parentheses or other grouping operator, that's not what's going on here!
BTW, the [[ ]] form that Korthrun used is very similar to [ ], but isn't supported by more basic shells. It does avoid some nasty syntax oddities with [ ], though, so if you're using bash it's a good way to go.
Also, note that $? gives the status of the last command executed, so it gets reset by every single command that executes. For example, this won't work:
scp ...
echo "scp's exit status was $?"
if [ $? -eq 0 ]; then # Don't do this!!!!
...because the if is then looking at the exit status of the echo command, not scp! If you need to do something like this, store the status in a variable:
scp ...
scpstatus=$?
echo "scp's exit status was $scpstatus"
if [ $scpstatus -eq 0 ]; then
I've been trying to learn the syntax of logic statements in bash, how to do if/else, pipes and stuff. I'm trying to build a bash script, but I fail miserably after 3 hours of not getting how this stuff works.
Now I need this little script, I'll try to explain it using a generalized code, or call it whatever you want. Here you go:
while variable THRESHOLD = 10
{
if netstat -anltp contains a line with port 25565
then set variable THRESHOLD to 0 and variable PROCNUM to the process number,
else add 1 to variable THRESHOLD
sleep 5 seconds
}
kill the process No. PROCNUM
restart the script
Basically, what it does is, that once the socket closes, after a few tries, it kills the process which was listening on that port.
I'm pretty sure it's possible, but I can't figure out how to do it properly. Mostly because I don't understand pipes and am not really familiar with grep. Thank you for your help, in advance.
Don't want be offensive, but if you can write a "generalized" program all you need is learn th syntax of the while, if for bash and read the man pages of the grep and kill and so on...
And the pipes are the same as in your garden. Having two things: tap and pond. You can fill your pond with many ways (e.g. with rain). Also, you can open your tap getting water. But if you want fill the pond with the water from a tap, need a pipe. That's all. Syntax:
tap | pond
the output from a tap
connect with a pipe
to the (input) of the pond
e.g.
netstat | grep
the output from a netstat
connect with a pipe
to the input of the grep
that's all magic... :)
About the syntax: You tagged your question as bash.
So googling for a bash while syntax will show to you, this Beginners Bash guide
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_09_02.html
to, and you can read about the if in the same website.
Simply can't believe than after 3 hours you cannot understand basic while and if syntax to write your program with a bash syntax - especially, when you able write an "generalized" program...
is is not to hard (with modifying the 1st example in the above page) to write:
THRESHOLD="0"
while [ $THRESHOLD -lt 10 ]
do
#do the IF here
THRESHOLD=$[$THRESHOLD+1]
done
and so on...
#!/bin/bash
# write a little function
function do_error {
echo "$#" 1>&2
exit 1
}
# make the user pass in the path to the executable
if [ "$1" == "" ]; then
do_error "USAGE: `basename $0` <path to your executable>"
fi
if [ ! -e $1 ]; then
do_error "Unable to find executable at $1"
fi
if [ ! -x $1 ]; then
do_error "$1 is not an executable"
fi
PROC="$1"
PROCNAME=`basename $PROC`
# forever
while [ 1 ]; do
# check whether the process is up
proc=`ps -ef | grep $PROCNAME 2>/dev/null`
# if it is not up, start it in the background (unless it's a daemon)
if [ "$proc" == "" ]; then
$PROC &
fi
# reinitialize the threshold
threshold=0
# as long as we haven't tried 10 time, continue trying
while [ threshold -lt 10 ]; do
# run netstat, look for port 25565, and see if the connection is established.
# it would be better to checks to make sure
# that the process we expect is the one that established the connection
output=`netstat -anp | grep 25565 | grep ESTABLISHED 2>/dev/null`
# if netstat found something, then our process was able to establish the connection
if [ "$output" != "" ]; then
threshold = 0
else
# increment the threshold
threshold=$((threshold + 1))
fi
# i would sleep for one second
sleep 1
done
kill -9 $PROCNUM
done