I am a novice in shell scripting.I have an unzip operation at the very beginning and then follow a sequence of steps.I would like to move to these steps only if the gunzip operation was successful.
towards that end i have a very simple script right now
#!/bin/bash
set -x -e
x=$(gunzip myfile.dat.gz)
echo "X is " $x
the output is
-bash-3.2$ ./unzipper.sh
++ gunzip myfile.dat.gz
+ x=
+ echo 'X is '
X is
I was expecting x to have a 0/non zero status code and based on that would have wanted to use a if condition to carry on with the next.
It doesn't look like this is correct.
Can you please help me with how to do this?
Most likely what you want to do is:
#!/bin/bash
if gunzip myfile.dat.gz; then
# What you want to do on success goes here
fi
At least, that is the simplest solution. set -e has a lot of subtle results; it's not recommended.
If you really want to save the status code of the gunzip, you can use $? immediately after the command:
#!/bin/bash
gunzip myfile.dat.gz
gunzip_status=$?
# Do some stuff whether or not gunzip succeeded
if ((gunzip_status==0)); then
# Do some stuff only if gunzip succeeded
else
# Do some stuff only if gunzip failed
fi
# What you want to do on success goes here
fi
Related
I am running quantum chemical calculations by providing the command molcas -f file.input. I now have need for putting the molcas -f into a script that also tails the last 100 lines of the generated file.log, for me to quickly confirm that everything finished the way it's supposed to. So I want to run the script run.sh:
#!/bin/bash
molcas -f [here read the file.input from command line]
tail -100 [here read the file.log]
The question is how I can make the script read the argument I give, and then find on its own the output file (which has the same filename, but with a different extension).
Follow-up
Say I have a bunch of numbered files file-1, file-2, ..., file-n. I would save time if I instead of running
./run.sh file-1.input file-1.log
I run
./run.sh n n
or
./run.sh n.input n.log
assuming that the actual filename and placement of the number n is given in the script. Can that be done?
With this code:
#!/bin/bash
molcas -f "$1"
tail -100 "$2"
You will need to execute the script run.sh as follows:
./run.sh file.input file.log
to be hornest I have/had no clue over molcas, so I jumed to this side to get basic understandings.
The syntax shoould look like this ...
#!/bin/bash
# waiting for input
read -p "Enter a filename (sample.txt): " FILE
# checking for existing file
if [ -e "$FILE" ]; then
read -p "Enter a command for moculas: " CMD
else
echo "Sorry, \"${FILE}\" was not found. Exit prgramm."
exit 1
fi
# I am not sure how this command works.
# maybe you have to edit this line by your self.
molcas $FILE -f "$CMD"
# checking for programm-errors
ERRNO=$?
if [ "$ERRNO" != "" ] && [ "$ERRNO" -gt 0 ]; then
echo "Molcas returned an error. Number: ${ERRNO}"
exit 1
fi
# cuts off the fileending (For example: sample.txt gets sample)
FILENAME="${FILE%.*}"
# checking other files
ERRFILE="${FILENAME}.err"
tail -n 100 $ERRFILE
LOGFILE="${FILENAME}.log"
tail -n 100 $LOGFILE
exit 0
I would have posted more, but its not clear what to do with this data.
Hope this helps a bit.
I'm wondering if I've done this right. I'm trying to learn BASH and really want to learn the "Best Practices" the first time, so I don't adopt the sloppy/easy way.
What I'm wondering, can I nest an IF/THEN statement like I've done below? Why or why not? Would the block below be served better by using an elif instead?
Lastly, I was hoping someone could shed some light for me on the use of "${foo}" and "$(bar)" ... curly braces or parenthesis? I've (so far) used curly braces when I'm defining a variable "foo='bar'" is later called as "${foo} and parenthesis when I'm capturing a command "foo=$(find . -type f -name bar)" would be called as "$foo" ... or maybe I'm just way off and doing the same thing twice, I don't know ... I'd love to hear what you've all got to say! :D
# Downloading the script bundle
echo "Lets get the script bundle and get to work!"
wget http://place.to.get/att.tar
# Logic switch, checking if the TAR bundle exists. If it does
# verify the MD5 Checksum (to prevent corruption).
# If verfied, then un-tar the bundle in our working directory
# otherwise, exit with an error code, otherwise
if [[ -f att.tar ]]
then
echo "Okay, we have the bundle, lets verify the checksum"
sum=$(md5sum /root/att/att.tar | awk '{print $1}')
if [[ $sum -eq "xxxxINSERT-CHECKSUM-HERExxxx" ]]
then
tar -xvf att.tar
else
clear
echo "Couldn't verify the MD5 Checksum, something went wrong" | tee /tmp/att.$time.log
sleep 0.5
exit 1;
fi
else
clear
echo "There was a problem getting the TAR bundle, exiting now ..." | tee /tmp/att.$time.log
sleep 0.5
exit 1;
fi
Overall comments
Nothing wrong with nested "if's," but early exit would be clearer
cut is cheaper than awk, but read is cheaper still
Simple string equality tests are marginally cheaper with "[" rather than "[["
Write error messages to STDERR
Use read and < <() rather than $( | cut -f1 -d' ') because it avoids a pipe and second fork/exec
Use functions
A simplified version
bail () {
clear
echo "${#}" | tee /tmp/att.${time}.log >&2
exit 1
}
# Downloading the script bundle
echo "Lets get the script bundle and get to work!" >&2
wget http://place.to.get/att.tar || bail "There was a problem getting the TAR bundle, exiting now ..."
sum=''
read sum rest < <(md5sum /root/att/att.tar)
[ $sum == "xxxxINSERT-CHECKSUM-HERExxxx" ] || bail "Couldn't verify the MD5 Checksum, something went wrong"
tar -xvf att.tar || bail "Extract failed"
I am trying to automate out a lot of our pre fs/db tasks and one thing that bugs me is not knowing whether or not a command i issue REALLY happened. I'd like a way to be able to watch for a return code of some sort from executing that command. Where if it fails to rm because of a permission denied or any error. to issue an exit..
If i have a shell script as such:
rm /oracle/$SAPSID/mirrlogA/cntrl/cntrl$SAPSID.ctl;
psuedo code could be something similar to..
rm /oracle/$SAPSID/mirrlogA/cntrl/cntrl$SAPSID.ctl;
if [returncode == 'error']
exit;
fi
how could i for example, execute that rm command and exit if its NOT rm'd. I will be adapting the answer to execute with multiple other types of commands such as sed -i -e, and cp and umount
edit:
Lets suppose i have a write protected file such as:
$ ls -lrt | grep protectedfile
-rwx------ 1 orasmq sapsys 0 Nov 14 12:39 protectedfile
And running the below script generates the following error because obviously theres no permissions..
rm: remove write-protected regular empty file `/tmp/protectedfile'? y
rm: cannot remove `/tmp/protectedfile': Operation not permitted
Here is what i worked out from your guys' answers.. is this the right way to do something like this? Also how could i dump the error rm: cannot remove /tmp/protectedfile': Operation not permitted` to a logfile?
#! /bin/bash
function log(){
//some logging code, simply writes to a file and then echo's out inpit
}
function quit(){
read -p "Failed to remove protected file, permission denied?"
log "Some log message, and somehow append the returned error message from rm"
exit 1;
}
rm /tmp/protectedfile || quit;
If I understand correctly what you want, just use this:
rm blah/blah/blah || exit 1
a possibility: a 'wrapper' so that you can retrieve the original commands stderr |and stdout?], and maybe also retry it a few times before giving up?, etc.
Here is a version that redirects both stdout and stderr
Of course you could not redirect stdout at all (and usually, you shouldn't, i guess, making the "try_to" function a bit more useful in the rest of the script!)
export timescalled=0 #external to the function itself
try_to () {
let "timescalled += 1" #let "..." allows white spaces and simple arithmetics
try_to_out="/tmp/try_to_${$}.${timescalled}"
#tries to avoid collisions within same script and/or if multiple script run in parrallel
zecmd="$1" ; shift ;
"$1" "$#" 2>"${try_to_out}.ERR" >"${try_to_out}.OUT"
try_to_ret=$?
#or: "$1" "$#" >"${try_to_out}.ERR" 2>&1 to have both in the same one
if [ "$try_to_ret" -ne "0" ]
then log "error $try_to_ret while trying to : '${zecmd} $#' ..." "${try_to_out}.ERR"
#provides custom error message + the name of the stderr from the command
rm -f "${try_to_out}.ERR" "${try_to_out}.OUT" #before we exit, better delete this
exit 1 #or exit $try_to_ret ?
fi
rm -f "${try_to_out}.ERR" "${try_to_out}.OUT"
}
it's ugly, but could help ^^
note that there are many things that could go wrong: 'timecalled' could become too high, the tmp file(s) could not be writable, zecmd could contain special caracters, etc...
Usualy some would use a command like this:
doSomething.sh
if [ $? -ne 0 ]
then
echo "oops, i did it again"
exit 1;
fi
B.T.W. searching for 'bash exit status' will give you already a lot of good results
I'm working on a script that has gotten so complex I want to include an easy option to update it to the most recent version. This is my approach:
set -o errexit
SELF=$(basename $0)
UPDATE_BASE=http://something
runSelfUpdate() {
echo "Performing self-update..."
# Download new version
wget --quiet --output-document=$0.tmp $UPDATE_BASE/$SELF
# Copy over modes from old version
OCTAL_MODE=$(stat -c '%a' $0)
chmod $OCTAL_MODE $0.tmp
# Overwrite old file with new
mv $0.tmp $0
exit 0
}
The script seems to work as intended, but I'm wondering if there might be caveats with this kind of approach. I just have a hard time believing that a script can overwrite itself without any repercussions.
To be more clear, I'm wondering, if, maybe, bash would read and execute the script line-by-line and after the mv, the exit 0 could be something else from the new script. I think I remember Windows behaving like that with .bat files.
Update: My original snippet did not include set -o errexit. To my understanding, that should keep me safe from issues caused by wget.
Also, in this case, UPDATE_BASE points to a location under version control (to ease concerns).
Result: Based on the input from these answers, I constructed this revised approach:
runSelfUpdate() {
echo "Performing self-update..."
# Download new version
echo -n "Downloading latest version..."
if ! wget --quiet --output-document="$0.tmp" $UPDATE_BASE/$SELF ; then
echo "Failed: Error while trying to wget new version!"
echo "File requested: $UPDATE_BASE/$SELF"
exit 1
fi
echo "Done."
# Copy over modes from old version
OCTAL_MODE=$(stat -c '%a' $SELF)
if ! chmod $OCTAL_MODE "$0.tmp" ; then
echo "Failed: Error while trying to set mode on $0.tmp."
exit 1
fi
# Spawn update script
cat > updateScript.sh << EOF
#!/bin/bash
# Overwrite old file with new
if mv "$0.tmp" "$0"; then
echo "Done. Update complete."
rm \$0
else
echo "Failed!"
fi
EOF
echo -n "Inserting update process..."
exec /bin/bash updateScript.sh
}
(At least it doesn't try to continue running after updating itself!)
The thing that makes me nervous about your approach is that you're overwriting the current script (mv $0.tmp $0) as it's running. There are a number of reasons why this will probably work, but I wouldn't bet large amounts that it's guaranteed to work in all circumstances. I don't know of anything in POSIX or any other standard that specifies how the shell processes a file that it's executing as a script.
Here's what's probably going to happen:
You execute the script. The kernel sees the #!/bin/sh line (you didn't show it, but I presume it's there) and invokes /bin/sh with the name of your script as an argument. The shell then uses fopen(), or perhaps open() to open your script, reads from it, and starts interpreting its contents as shell commands.
For a sufficiently small script, the shell probably just reads the whole thing into memory, either explicitly or as part of the buffering done by normal file I/O. For a larger script, it might read it in chunks as it's executing. But either way, it probably only opens the file once, and keeps it open as long as it's executing.
If you remove or rename a file, the actual file is not necessarily immediately erased from disk. If there's another hard link to it, or if some process has it open, the file continues to exist, even though it may no longer be possible for another process to open it under the same name, or at all. The file is not physically deleted until the last link (directory entry) that refers to it has been removed, and no processes have it open. (Even then, its contents won't immediately be erased, but that's going beyond what's relevant here.)
And furthermore, the mv command that clobbers the script file is immediately followed by exit 0.
BUT it's at least conceivable that the shell could close the file and then re-open it by name. I can't think of any good reason for it to do so, but I know of no absolute guarantee that it won't.
And some systems tend to do stricter file locking that most Unix systems do. On Windows, for example, I suspect that the mv command would fail because a process (the shell) has the file open. Your script might fail on Cygwin. (I haven't tried it.)
So what makes me nervous is not so much the small possibility that it could fail, but the long and tenuous line of reasoning that seems to demonstrate that it will probably succeed, and the very real possibility that there's something else I haven't thought of.
My suggestion: write a second script whose one and only job is to update the first. Put the runSelfUpdate() function, or equivalent code, into that script. In your original script, use exec to invoke the update script, so that the original script is no longer running when you update it. If you want to avoid the hassle of maintaining, distributing, and installing two separate scripts. you could have the original script create the update script with a unique in /tmp; that would also solve the problem of updating the update script. (I wouldn't worry about cleaning up the autogenerated update script in /tmp; that would just reopen the same can of worms.)
Yes, but ... I would recommend you keep a more layered version of your script's history, unless the remote host can also perform version-control with histories. That being said, to respond directly to the code you have posted, see the following comments ;-)
What happens to your system when wget has a hiccup, quietly overwrites part of your working script with only a partial or otherwise corrupt copy? Your next step does a mv $0.tmp $0 so you've lost your working version. (I hope you have it in version control on the remote!)
You can check to see if wget returns any error messages
if ! wget --quiet --output-document=$0.tmp $UPDATE_BASE/$SELF ; then
echo "error on wget on $UPDATE_BASE/$SELF"
exit 1
fi
Also, Rule-of-thumb tests will help, i.e.
if (( $(wc -c < $0.tmp) >= $(wc -c < $0) )); then
mv $0.tmp $0
fi
but are hardly foolproof.
If your $0 could windup with spaces in it, better to surround all references like "$0".
To be super-bullet proof, consider checking all command returns AND that Octal_Mode has a reasonable value
OCTAL_MODE=$(stat -c '%a' $0)
case ${OCTAL_MODE:--1} in
-[1] )
printf "Error : OCTAL_MODE was empty\n"
exit 1
;;
777|775|755 ) : nothing ;;
* )
printf "Error in OCTAL_MODEs, found value=${OCTAL_MODE}\n"
exit 1
;;
esac
if ! chmod $OCTAL_MODE $0.tmp ; then
echo "error on chmod $OCTAL_MODE %0.tmp from $UPDATE_BASE/$SELF, can't continue"
exit 1
fi
I hope this helps.
Very late answer here, but as I just solved this too, I thought it might help someone to post the approach:
#!/usr/bin/env bash
#
set -fb
readonly THISDIR=$(cd "$(dirname "$0")" ; pwd)
readonly MY_NAME=$(basename "$0")
readonly FILE_TO_FETCH_URL="https://your_url_to_downloadable_file_here"
readonly EXISTING_SHELL_SCRIPT="${THISDIR}/somescript.sh"
readonly EXECUTABLE_SHELL_SCRIPT="${THISDIR}/.somescript.sh"
function get_remote_file() {
readonly REQUEST_URL=$1
readonly OUTPUT_FILENAME=$2
readonly TEMP_FILE="${THISDIR}/tmp.file"
if [ -n "$(which wget)" ]; then
$(wget -O "${TEMP_FILE}" "$REQUEST_URL" 2>&1)
if [[ $? -eq 0 ]]; then
mv "${TEMP_FILE}" "${OUTPUT_FILENAME}"
chmod 755 "${OUTPUT_FILENAME}"
else
return 1
fi
fi
}
function clean_up() {
# clean up code (if required) that has to execute every time here
}
function self_clean_up() {
rm -f "${EXECUTABLE_SHELL_SCRIPT}"
}
function update_self_and_invoke() {
get_remote_file "${FILE_TO_FETCH_URL}" "${EXECUTABLE_SHELL_SCRIPT}"
if [ $? -ne 0 ]; then
cp "${EXISTING_SHELL_SCRIPT}" "${EXECUTABLE_SHELL_SCRIPT}"
fi
exec "${EXECUTABLE_SHELL_SCRIPT}" "$#"
}
function main() {
cp "${EXECUTABLE_SHELL_SCRIPT}" "${EXISTING_SHELL_SCRIPT}"
# your code here
}
if [[ $MY_NAME = \.* ]]; then
# invoke real main program
trap "clean_up; self_clean_up" EXIT
main "$#"
else
# update myself and invoke updated version
trap clean_up EXIT
update_self_and_invoke "$#"
fi
Could someone help me understand the following piece of code which is deciding on the start and end dates to pick data out of a db.
# Get the current time as the stop time.
#
stoptime=`date +"%Y-%m-%d %H:00"`
if test $? -ne 0
then
echo "Failed to get the date"
rm -f $1/.optpamo.pid
exit 4
fi
#
# Read the lasttime file to get the start time
#
if test -f $1/optlasttime
then
starttime=`cat $1/optlasttime`
# if the length of the chain is zero
# (lasttime is empty) It is updated properly
# (and I wait for the following hour)
if test -z "$starttime"
then
echo "Empty file lasttime"
echo $stoptime > $1/optlasttime
rm -f $1/.optpamo.pid
exit 5
fi
else
# If lasttime does not exist I create, it with the present date
# and I wait for the following hour
echo "File lasttime does not exist"
echo $stoptime > $1/optlasttime
rm -f $1/.optpamo.pid
exit 6
fi
Thanks
The script checks to see if there's a non-empty file named optlasttime in the directory specified as an argument ($1). If so, the script exits successfully (status 0). If the file doesn't exist or is empty, the current hour formatted as 2010-01-07 14:00 is written to the file, another file named .optpamo.pid is deleted from the argument directory and the script exits unsuccessfully (status 5 or 6).
This script is obviously a utility being called by some outer process, to which you need to refer for full understanding.
1.) Sets stop time to current time
2.) Checks if file $1/optlasttime exists (where $1 is passed into the script)
a.) if $1/optlasttime exists it checks the contents of the file (which it is assumed that if it does have contents it is a timestamp)
b.) if $1/optlasttime does not exist it populates the $1/optlasttime file with the stoptime.
I copied and pasted a small snippet of this into a file I called test.ksh
stoptime=`date +"%Y-%m-%d %H:00"`
if test $? -ne 0
then
echo "Failed to get the date"
rm -f $1/.optpamo.pid
exit 4
fi
Then I ran it at the commandline, like so:
zhasper#berens:~$ ksh -x ./temp.ksh
+ date '+%Y-%m-%d %H:00'
+ stoptime='2010-01-08 18:00'
+ test 0 -ne 0
The -x flag to ksh makes it print out each commandline, in full, as it executes. Comparing what you see here with the snippet of shell script above should tell you something about how ksh is interpreting the file.
If you run this over the whole file you should get a good feel for what it's doing.
To learn more, you can read man ksh, or search for ksh scripting tutorial online.
Together, these three things should help you learn a lot more than us simply telling you what the script does.