Running another script in bash script while loop runs but the loop breaks!
N.B. The script I mentioned just loops over files in current directory and just run mpirun.
Here's my bash script:
#!/bin/bash
np="$1"
bin="$2"
ref="$3"
query="$4"
word_size="$5"
i=1;
input="$query"
while read line; do
echo $line
if [[ "${line:0:1}" == ">" ]] ; then
header="$line"
echo "$header" >> seq_"${i}".fasta
else
seq="$line"
echo "$seq" >> seq_"${i}".fasta
if ! (( i % 5)) ; then
./run.sh $np $bin $ref $word_size
^^^^^^^^
#for filename in *.fasta; do
# mpirun -np "${np}" "${bin}" -d "${ref}" -ql "${filename}" -k "${word_size}" -b > log
# rm $filename
#done
fi
((i++))
fi
done < $input
The problem is that your run.sh script is passing no parameters to mpirun. That script passes the variables ${np} ${bin} ${ref} ${filename} ${word_size} to mpirun, but those variables are local to your main script and are undefined in run.sh. You could export those variables in the main script so that they are available to all child processes, but a better solution would be to use positional parameters in run.sh:
for filename in *.fasta; do
mpirun -np "${1}" "${2}" -d "${3}" -ql "${4}" -k "${5}" -b > log
rm $filename
done
I don't know about mpirun, but if you have anything inside your loop that reads from stdin, the loop will break.
I am using the bash trap to make sure one function runs at any cost. I know trap is not specific to exitO or 1. Here is what I have done.
#!/bin/bash
set -e
#array to store server and deployed status
declare -A server_deployed
#path to file containing the server inventory
readonly filepath="/var/jenkins_home/workspace/server_list.txt"
#array to store to list of IPS
declare -A result #associative array
if [[ $TARGET == "ALL" ]]; then
while read line ; do
server_name=` echo $line | cut -d= -f1 `
result+=(["$server_name"]=${line#*=})
done < $filepath
else
singleserver=`cat $filepath | grep "$TARGET"`
server_name=`echo $singleserver| cut -d= -f1 ` # get the servername
serverip=`echo $singleserver| cut -d= -f2 ` # get tje server ip
result+=(["$server_name"]=$serverip) # gets the ip which is after equalto
fi
#function to send slack notification everytime
function sendMessage(){
for sd in "${!server_deployed[#]}"
do
echo "#########################################################"
echo "Sucecssfully deployed on $sd"
echo "#########################################################"
done
#let find the unsuccess list ,for which we need to find the array diff
for server_name in "${!result[#]}"
do
for sd in "${!server_deployed[#]}"
do
if [[ "$server_name" != "$sd" ]]; then
echo "Failed to deploy on $server_name"
fi
done
done
}
trap "sendMessage" INT EXIT
for server in "${!result[#]}"
do
upstream="MBM-TEST-ADMIN-BUILD"
#bundle filename
bundle="mbm_admin_dist"
name=$server
instance=${result[$name]}
#------copying the hash.php in build job to the deployed server home location
# ssh and move to desired location
sudo scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /tmp/test/admin_hash.json ubuntu#$instance:/home/ubuntu/
mbmcms_workspace="/var/jenkins_home/workspace/$upstream"
cd $mbmcms_workspace
#remove if exist
if [ -e "$bundle.zip" ]
then
echo "Already Exist so Removing it First !!"
rm -f $bundle.zip
fi
#check if the dist folder exist that comes from successful build
if [ ! -d "dist" ]; then
echo "--------------------------------------------------------------"
echo "The dist folder does not exist, Please run BUILD job First !!"
echo "--------------------------------------------------------------"
exit 1
fi
#zip the file from the TEST & Build jobs
echo "starting to zip mbm-admin dist file created after test and build success!!"
zip --symlinks -x *.git* -r $bundle ./dist
#---------------------------------------------------------------
# copy development.php if non-production else copy production.php
# the file goes to the view/cms/
#---------------------------------------------------------------
sudo scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null development.php ubuntu#$instance:/home/ubuntu
sudo scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null production.php ubuntu#$instance:/home/ubuntu
#copy the bundle file to instance
scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -r $bundle.zip ubuntu#$instance:/home/ubuntu/
ssh -tt -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu#$instance '
SOURCE_FOLDER='/home/ubuntu/mbm-admin/dist/'
DESTINATION_FOLDER='/home/ubuntu/wpdata/plugins/manager/app/views/admin/admin_console/dist/'
rm -rf mbm-admin
mkdir -p mbm-admin
mv mbm_admin_dist.zip mbm-admin
cd mbm-admin
unzip mbm_admin_dist.zip
rm -f mbm_admin_dist.zip
sudo rsync -arz --force --delete --progress $SOURCE_FOLDER $DESTINATION_FOLDER
#moving the copied file to the /app/views/admin/admin_console
sudo mv /home/ubuntu/development.php /home/ubuntu/wpdata/plugins/manager/app/views/admin/admin_console
sudo mv /home/ubuntu/production.php /home/ubuntu/wpdata/plugins/manager/app/views/admin/admin_console
#moving the hash.php to the desired location
sudo cp -fv /home/ubuntu/admin_hash.json /home/ubuntu/wpdata/plugins/manager/app/views/admin/admin_console/hash.json
'
server_deployed+=(["$server"]="SUCCESS")
done
Here is the trap , as seen the script section
trap "sendMessage" INT EXIT
I need to run the sendMessage function whenever some exit code is
detected or the program exits due to any error.
Problem:
When I put the exit 1 at the end of the script the function gets called by the trap but suppose, If I put it somewhere in the middle or exactly after the main for loop starts the trap does not catch the exit code.
What am I not understanding or missing exactly in here?
I'm working on a script to automate the creation of a .gitconfig file.
This is my main script that calls a function which in turn execute another file.
dotfile.sh
COMMAND_NAME=$1
shift
ARG_NAME=$#
set +a
fail() {
echo "";
printf "\r[${RED}FAIL${RESET}] $1\n";
echo "";
exit 1;
}
set -a
sub_setup() {
info "This may overwrite existing files in your computer. Are you sure? (y/n)";
read -p "" -n 1;
echo "";
if [[ $REPLY =~ ^[Yy]$ ]]; then
for ARG in $ARG_NAME; do
local SCRIPT="~/dotfiles/setup/${ARG}.sh";
[ -f "$SCRIPT" ] && echo "Applying '$ARG'" && . "$SCRIPT" || fail "Unable to find script '$ARG'";
done;
fi;
}
case $COMMAND_NAME in
"" | "-h" | "--help")
sub_help;
;;
*)
CMD=${COMMAND_NAME/*-/}
sub_${CMD} $ARG_NAME 2> /dev/null;
if [ $? = 127 ]; then
fail "'$CMD' is not a known command or has errors.";
fi;
;;
esac;
git.sh
git_config() {
if [ ! -f "~/dotfiles/git/gitconfig_template" ]; then
fail "No gitconfig_template file found in ~/dotfiles/git/";
elif [ -f "~/dotfiles/.gitconfig" ]; then
fail ".gitconfig already exists. Delete the file and retry.";
else
echo "Setting up .gitconfig";
GIT_CREDENTIAL="cache"
[ "$(uname -s)" == "Darwin" ] && GIT_CREDENTIAL="osxkeychain";
user " - What is your GitHub author name?";
read -e GIT_AUTHORNAME;
user " - What is your GitHub author email?";
read -e GIT_AUTHOREMAIL;
user " - What is your GitHub username?";
read -e GIT_USERNAME;
if sed -e "s/AUTHORNAME/$GIT_AUTHORNAME/g" \
-e "s/AUTHOREMAIL/$GIT_AUTHOREMAIL/g" \
-e "s/USERNAME/$GIT_USERNAME/g" \
-e "s/GIT_CREDENTIAL_HELPER/$GIT_CREDENTIAL/g" \
"~/dotfiles/git/gitconfig_template" > "~/dotfiles/.gitconfig"; then
success ".gitconfig has been setup";
else
fail ".gitconfig has not been setup";
fi;
fi;
}
git_config
In the console
$ ./dotfile.sh --setup git
[ ?? ] This may overwrite existing files in your computer. Are you sure? (y/n)
y
Applying 'git'
Setting up .gitconfig
[ .. ] - What is your GitHub author name?
Then I cannot see what I'm typing...
At the bottom of dotfile.sh, I redirect any error that occurs during my function call to /dev/null. But I should normally see what I'm typing. If I remove 2> /dev/null from this line sub_${CMD} $ARG_NAME 2> /dev/null;, it works!! But I don't understand why.
I need this line to prevent my script to echo an error in case my command doesn't exists. I only want my own message.
e.g.
$ ./dotfile --blahblah
./dotfiles: line 153: sub_blahblah: command not found
[FAIL] 'blahblah' is not a known command or has errors
I really don't understand why the input in my sub script is redirected to /dev/null as I mentioned only stderr to be redirected to /dev/null.
Thanks
Do you need the -e option in your read statements?
I did a quick test in an interactive shell. The following command does not echo characters :
read -e TEST 2>/dev/null
The following does echo the characters
read TEST 2>/dev/null
I'm running it under MacOS El Capitan 10.10.6
among all commands to get my current dir (path I'm running my script from) only this works for me:
FILES="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
But it's not going to work if the folder has whitespace in it's name (aka: "folder name")
How to fix this?
Thank you! )
Update: added a script:
#!/bin/bash
function check ()
{
oldsize=`wc -c <"$1"`
sleep 1
newsize=`wc -c <"$1"`
while [ "$oldsize" -lt "$newsize" ]
do
echo "Not yet..."
oldsize=`wc -c <"$1"`
sleep 1
newsize=`wc -c <"$1"`
done
if [ "$oldsize" -eq "$newsize" ]
then
echo "The file has been copied completely."
fi
}
FILES="$(dirname "${BASH_SOURCE[0]}")/*"
function main
{
for f in $FILES
do
if [[ "$f" =~ \.mkv$ ]];
then
#/////////////////////////////////////////////////////////////////////
check "$f"
(( count = count + 1 ))
g="${f/mkv/avi}"
#LOG_FILE="${g/avi/log}"
#exec > >(tee -a "${LOG_FILE}" )
#exec 2> >(tee -a "${LOG_FILE}" >&2)
now="$(date)"
printf "Current date and time %s\n" "$now"
echo "Processing $f file..."
#avconv -i "${f}" -map 0:0 -map 0:1 -codec copy -sn "${g}"
avconv -i "$f" -map 0 -codec copy "$g"
if [ $? -eq 0 ]; then
echo OK
rm "$f"
else
echo FAIL
rm "$g"
#rm "$LOG_FILE"
return
fi
fi
#/////////////////////////////////////////////////////////////////////
done
}
############
count=0
############
main
if (($count > 0)); then
open "$(dirname "${BASH_SOURCE[0]}")"
fi
exit
I am using Mac OS X 10.11.6, and I have a directory $HOME/tmp. From there, I executed:
$ cd $HOME/tmp
$ pwd
/Users/jleffler/tmp
$ mkdir -p "Spaced Out Directory "/bin
$ export PATH="$PATH:$PWD/$_"
$ cat <<'EOF' > Spaced\ Out\ \ Directory\ \ \ /bin/gorblinsky.sh
> #!/bin/bash
>
> echo "PWD=$PWD"
> DIR="$(dirname "${BASH_SOURCE[0]}")"
> echo "DIR=$DIR"
> cd "$DIR"
> pwd
> echo "PWD=$PWD"
> EOF
$ chmod +x Spaced\ Out\ \ Directory\ \ \ /bin/gorblinsky.sh
$ gorblinsky.sh
PWD=/Users/jleffler/tmp
DIR=/Users/jleffler/tmp/Spaced Out Directory /bin
/Users/jleffler/tmp/Spaced Out Directory /bin
PWD=/Users/jleffler/tmp/Spaced Out Directory /bin
$
This shows that the command $(dirname "${BASH_SOURCE[0]}") can determine the name of the directory where the source for the command is stored.
If the script was going to use the variable $DIR to specify file names, you'd need to be careful (very careful) to ensure it is always properly quoted.
For example:
cp "$DIR/gorblinksky.h" "$HOME/tmp/cobbled together name"
Modern style is to always (double) quote all variable references, even when there's nothing in them that needs protecting (see shellcheck.net for example — and Google Shell Style Guide). I'm old-school enough not to put quotes around names that can't contain spaces or metacharacters, but I guess that is just old-fashioned. For example, I shell-checked a script for playing with RCS version numbers, and it doesn't quote variables containing dotted strings of digits (9.19.2.24 — could be an IBM IPv4 address too) and I was told off for not quoting them, though the file names were already protected with quotes.
readlink -f does not exist on MacOS. The only working solution for Mac OS I managed to find on the net goes like this:
if [[ $(echo $0 | awk '/^\//') == $0 ]]; then
ABSPATH=$(dirname $0)
else
ABSPATH=$PWD/$(dirname $0)
fi
Can anyone suggest anything more elegant to this seemingly trivial task?
Another (also rather ugly) option:
ABSPATH=$(cd "$(dirname "$0")"; pwd -P)
From pwd man page,
-P Display the physical current working directory (all symbolic links resolved).
Get absolute path of shell script
Dug out some old scripts from my .bashrc, and updated the syntax a bit, added a test suite.
Supports
source ./script (When called by the . dot operator)
Absolute path /path/to/script
Relative path like ./script
/path/dir1/../dir2/dir3/../script
When called from symlink
When symlink is nested eg) foo->dir1/dir2/bar bar->./../doe doe->script
When caller changes the scripts name
It has been tested and used in real projects with success, however there may be corner cases I am not aware of.
If you were able to find such a situation, please let me know.
(For one, I know that this does not run on the sh shell)
Code
pushd . > /dev/null
SCRIPT_PATH="${BASH_SOURCE[0]}";
while([ -h "${SCRIPT_PATH}" ]) do
cd "`dirname "${SCRIPT_PATH}"`"
SCRIPT_PATH="$(readlink "`basename "${SCRIPT_PATH}"`")";
done
cd "`dirname "${SCRIPT_PATH}"`" > /dev/null
SCRIPT_PATH="`pwd`";
popd > /dev/null
echo "srcipt=[${SCRIPT_PATH}]"
echo "pwd =[`pwd`]"
Known issuse
Script must be on disk somewhere, let it be over a network.
If you try to run this script from a PIPE it will not work
wget -o /dev/null -O - http://host.domain/dir/script.sh |bash
Technically speaking, it is undefined.
Practically speaking, there is no sane way to detect this.
Test case used
And the current test case that check that it works.
#!/bin/bash
# setup test enviroment
mkdir -p dir1/dir2
mkdir -p dir3/dir4
ln -s ./dir1/dir2/foo bar
ln -s ./../../dir3/dir4/test.sh dir1/dir2/foo
ln -s ./dir1/dir2/foo2 bar2
ln -s ./../../dir3/dir4/doe dir1/dir2/foo2
cp test.sh ./dir1/dir2/
cp test.sh ./dir3/dir4/
cp test.sh ./dir3/dir4/doe
P="`pwd`"
echo "--- 01"
echo "base =[${P}]" && ./test.sh
echo "--- 02"
echo "base =[${P}]" && `pwd`/test.sh
echo "--- 03"
echo "base =[${P}]" && ./dir1/dir2/../../test.sh
echo "--- 04"
echo "base =[${P}/dir3/dir4]" && ./bar
echo "--- 05"
echo "base =[${P}/dir3/dir4]" && ./bar2
echo "--- 06"
echo "base =[${P}/dir3/dir4]" && `pwd`/bar
echo "--- 07"
echo "base =[${P}/dir3/dir4]" && `pwd`/bar2
echo "--- 08"
echo "base =[${P}/dir1/dir2]" && `pwd`/dir3/dir4/../../dir1/dir2/test.sh
echo "--- 09"
echo "base =[${P}/dir1/dir2]" && ./dir1/dir2/test.sh
echo "--- 10"
echo "base =[${P}/dir3/dir4]" && ./dir3/dir4/doe
echo "--- 11"
echo "base =[${P}/dir3/dir4]" && ./dir3/dir4/test.sh
echo "--- 12"
echo "base =[${P}/dir3/dir4]" && `pwd`/dir3/dir4/doe
echo "--- 13"
echo "base =[${P}/dir3/dir4]" && `pwd`/dir3/dir4/test.sh
echo "--- 14"
echo "base =[${P}/dir3/dir4]" && `pwd`/dir1/dir2/../../dir3/dir4/doe
echo "--- 15"
echo "base =[${P}/dir3/dir4]" && `pwd`/dir1/dir2/../../dir3/dir4/test.sh
echo "--- 16"
echo "base s=[${P}]" && source test.sh
echo "--- 17"
echo "base s=[${P}]" && source `pwd`/test.sh
echo "--- 18"
echo "base s=[${P}/dir1/dir2]" && source ./dir1/dir2/test.sh
echo "--- 19"
echo "base s=[${P}/dir3/dir4]" && source ./dir1/dir2/../../dir3/dir4/test.sh
echo "--- 20"
echo "base s=[${P}/dir3/dir4]" && source `pwd`/dir1/dir2/../../dir3/dir4/test.sh
echo "--- 21"
pushd . >/dev/null
cd ..
echo "base x=[${P}/dir3/dir4]"
./`basename "${P}"`/bar
popd >/dev/null
PurpleFox aka GreenFox
Also note that homebrew's (http://brew.sh) coreutils package includes realpath (link created in/opt/local/bin).
$ realpath bin
/Users/nhed/bin
Using bash I suggest this approach. You first cd to the directory, then you take the current directory using pwd. After that you must return to the old directory to ensure your script does not create side effects to an other script calling it.
cd "$(dirname -- "$0")"
dir="$PWD"
echo "$dir"
cd - > /dev/null
This solution is safe with complex path. You will never have troubles with spaces or special charaters if you put the quotes.
Note: the /dev/null is require or "cd -" print the path its return to.
If you don't mind using perl:
ABSPATH=$(perl -MCwd=realpath -e "print realpath '$0'")
Can you try something like this inside your script?
echo $(pwd)/"$0"
In my machine it shows:
/home/barun/codes/ns2/link_down/./test.sh
which is the absolute path name of the shell script.
I've found this to be useful for symlinks / dynamic links - works with GNU readlink only though (because of the -f flag):
# detect if GNU readlink is available on OS X
if [ "$(uname)" = "Darwin" ]; then
which greadlink > /dev/null || {
printf 'GNU readlink not found\n'
exit 1
}
alias readlink="greadlink"
fi
# create a $dirname variable that contains the file dir
dirname=$(dirname "$(readlink -f "$0")")
# use $dirname to find a relative file
cat "$dirname"/foo/bar.txt
this is what I use, may need a tweak here or there
abspath ()
{
case "${1}" in
[./]*)
local ABSPATH="$(cd ${1%/*}; pwd)/${1##*/}"
echo "${ABSPATH/\/\///}"
;;
*)
echo "${PWD}/${1}"
;;
esac
}
This is for any file - and of curse you can just invoke it as abspath ${0}
The first case deals with relative paths by cd-ing to the path and letting pwd figure it out
The second case is for dealing with a local file (where the ${1##/} would not have worked)
This does NOT attempt to undo symlinks!
This works as long as it's not a symlink, and is perhaps marginally less ugly:
ABSPATH=$(dirname $(pwd -P $0)/${0#\.\/})
If you're using ksh, the ${.sh.file} parameter is set to the absolute pathname of the script. To get the parent directory of the script: ${.sh.file%/*}
I use the function below to emulate "readlink -f" for scripts that have to run on both linux and Mac OS X.
#!/bin/bash
# This was re-worked on 2018-10-26 after der#build correctly
# observed that the previous version did not work.
# Works on both linux and Mac OS X.
# The "pwd -P" re-interprets all symlinks.
function read-link() {
local path=$1
if [ -d $path ] ; then
local abspath=$(cd $path; pwd -P)
else
local prefix=$(cd $(dirname -- $path) ; pwd -P)
local suffix=$(basename $path)
local abspath="$prefix/$suffix"
fi
if [ -e $abspath ] ; then
echo $abspath
else
echo 'error: does not exist'
fi
}
# Example usage.
while (( $# )) ; do
printf '%-24s - ' "$1"
read-link $1
shift
done
This is the output for some common Mac OS X targets:
$ ./example.sh /usr/bin/which /bin/which /etc/racoon ~/Downloads
/usr/bin/which - /usr/bin/which
/bin/which - error: does not exist
/etc/racoon - /private/etc/racoon
/Users/jlinoff/Downloads - /Users/jlinoff/Downloads
The is the output for some linux targets.
$ ./example.sh /usr/bin/which /bin/whichx /etc/init.d ~/Downloads
/usr/bin/which - /usr/bin/which
/bin/whichx - error: does not exist
/etc/init.d - /etc/init.d
/home/jlinoff/Downloads - /home/jlinoff/Downloads